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Abstract 

The  Army  is  concerned  with  maintaining  safe  operations  in  light  of  increasing  op¬ 
erational  demands.  The  Army  Safety  Center’s  goal,  as  approved  by  the  Under  Secretary 
of  Defense  for  Acquisitions  and  Technology,  is  to  reduce  accident  severity  by  50%  in  the 
next  decade.  The  Safety  Center  chartered  the  Aviation  Safety  Investment  Strategy  Team 
to  evaluate  accidents  to  determine  their  hazards,  or  contributing  conditions,  and  their 
controls,  or  reduction  measures.  This  study  specifically  targets  these  force-modernized 
aircraft:  AH-64  Apache,  CH/MH-47  Chinook,  OH-58D  Kiowa  Warrior,  and  UH-60  Black 
Hawk. 

This  research  takes  a  look  at  selecting  the  best  portfolios  of  controls  to  minimize 
aviation  accident  severity.  The  accidents  are  simulated  using  Monte  Carlo  techniques. 
Value-Focused  Thinking  techniques  evaluate  the  severity  of  accidents  generated  by  the 
simulation.  The  optimization  is  approached  using  a  knapsack  heuristic.  Insights  into 
selecting  the  best  sets  of  controls  aid  decision  makers  when  determining  the  portfolios 
with  the  best  Percent  Severity  Reduction  given  budget  considerations. 


SELECTING  OPTIMAL  CONTROL  PORTFOLIOS 
TO  IMPROVE  ARMY  AVIATION  SAFETY 


I.  Introduction 

By  law,  The  Army  is  tasked  to  defend  the  United  States  and  its  territories;  sup¬ 
port  national  policies  and  objectives;  and  defeat  nations  responsible  for  aggression  that 
endanger  the  peace  and  security  of  the  United  States  (33).  Accomplishing  this  goal,  while 
minimizing  the  cost  to  its  soldiers  and  to  civilians,  requires  that  commanders  account  for 
risk  when  making  decisions.  Conscientious  soldiers  and  leaders  plan  for  uncertain  hazards 
and  seek  to  eliminate  accidents  both  during  times  of  peace,  as  well  as  in  combat.  Because 
accidents  have  historically  killed  more  soldiers  than  enemy  fibre,  and  these  losses  take  their 
toll  on  effectiveness,  morale  and  mission-readiness,  this  issue  cannot  be  ignored.  Identi¬ 
fying,  assessing,  and  controlling  these  risks  is  the  purpose  of  the  risk  management  process 
(8:1-1, 2).  The  Under-Secretary  of  Defense  for  Acquisition  and  Technology  admonished 
the  services  to  “...achieve  a  three  sigma  reduction  in  Class  A  accident  rate  in  five  years” 
(13).  The  Army  recognizes  the  significance  of  risk  management;  consequently,  the  Army 
relies  on  the  U.S.  Army  Safety  Center  (USASC)  to  maintain  the  Risk  Management  In¬ 
formation  System,  as  well  as  all  Army  accident  information,  and  evaluate  safety-control 
related  issues. 

The  number  of  Army  aviation  mishaps  have  increased  in  the  past  decade,  attracting 
public  attention  and  demanding  reasonable  solutions.  Increasing  mission  frequency  and 
complexity  compounded  by  decreasing  resources  may  have  contributed  significantly  to  this 
rise  in  accidents.  As  the  ops-tempo  increases,  more  attention  must  be  directed  toward  the 
hazards,  the  system  inadequacies  contributing  to  accidents. 

The  Army  has  chosen  to  address  its  aviation  hazards  and  controls  through  the  for¬ 
mation  of  the  Aviation  Safety  Investment  Strategy  Team  (ASIST),  a  group  of  safety  and 
aviation  experts  who  analyzed  accident  investigation  information  and  identified  potential 
hazards  and  the  controls  that  reduce  them  (13:1).  ASIST  proposed  a  strategy  that  the 
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Army  Vice  Chief  of  Staff  endorsed  in  April  1999:  to  reduce  the  total  annual  cost  of  aviation 
accidents;  to  reduce  the  rate  of  fatal  and  disabling  injuries;  and  to  decrease  the  accident 
rate,  all  by  50%  in  the  next  decade.  Since  the  establishment  of  this  strategy,  several 
aircraft  have  been  analyzed  and  the  accident  rate  for  FY00  is  15.9%  less  than  FY99  (34). 
Nonetheless,  meeting  the  50%  reduction  goal  across  the  board  is  ambitious  and  requires 
proper  analyses.  Risk  Management  enables  the  development,  fielding,  and  employment 
of  the  Total  Army  Force.  Five  steps  have  been  identified  to  conduct  an  adequate  risk 
management  program  (8:2-0): 

Step  1:  Identify  hazards 

Step  2:  Assess  hazards  to  determine  risks 

Step  3:  Develop  controls  to  make  risk  decisions 

Step  4:  Implement  controls 

Step  5:  Supervise  and  evaluate 

ASIST  has  and  will  continue  to  successfully  complete  steps  1,  2  and  3  for  each 
individual  airframe.  Step  3  can  be  broken  into  two  parts,  developing  the  controls  and 
making  the  risk  decisions.  The  best  controls  have  been  suggested  by  ASIST;  this  research 
focuses  on  the  best  sets  of  controls,  or  portfolios  of  controls,  for  the  Army  to  select. 

Risk  management  enhances  the  decision  making  process  by  providing  the  information 
required  to  make  informed  decisions  and  to  identify  control  measures  in  areas  that  lack 
specific  standards.  Supplementing  decision  making  resources,  risk  management  enables 
a  decision  maker  to  provide  reasonable,  defensible  alternatives  (8:1-4).  The  concept  of 
Value-Focused  Thinking  (VFT)  can  be  brought  to  bear  on  this  problem,  VFT  applies  the 
foundational  ideas  of  risk  management,  addressing  first  what  is  important  and  secondly 
how  to  achieve  it.  A  VFT  approach  puts  responsibility  for  identifying  important  values 
in  the  hands  of  the  leaders;  who  feel  ownership  and  are  able  to  defend  and  trust  solutions 
based  on  their  voiced  preferences  (18:92) 

For  two  years  the  USASC  has  seen  benefit  from  AFIT  research;  Sperling’s  VFT  study 
and  initial  value  model  inspired  USASC,  in  part,  to  form  ASIST  (30).  Gallan  contributed 
the  improved  Severity  of  Losses  model  to  the  effort  (13).  The  research  described  herein 
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follows  on  those  efforts,  addressing  the  selection  of  an  optimal  region  for  portfolios  of  con¬ 
trols  given  certain  constraints,  such  as  available  budget.  A  systematic  way  of  evaluating 
the  effectiveness  of  an  individual  control  or  a  set  of  controls  was  established  and  accepted, 
but  selecting  an  optimal  set  of  controls  has  not  been  done  previously.  Using  integer  pro¬ 
gramming  and  a  direct-search  driven  knapsack  approach,  an  optimal  region  for  selecting 
portfolios  of  controls  is  identified.  Given  the  inherent  uncertainty,  coupled  with  the  combi¬ 
natorial  size  of  the  problem,  this  heuristic-based  method  identifies  the  region  for  multiple, 
feasible,  reliable  results  leading  to  relevant  insights  into  the  system,  aiding  decision  makers 
at  the  Safety  Center. 

1.1  Background 

Meeting  the  operational  demands  of  mission  readiness  while  keeping  up  with  tech¬ 
nological  advances  in  the  evolving  environment  faced  by  today’s  Army  requires  constant 
vigilance  and  effective  safety  analysis.  With  a  strong  foundation  in  Army  Field  Manual 
100-14,  the  Army  approaches  safety  from  a  risk  management  viewpoint.  In  this  context, 
risk  management  is  a  process  for  making  decisions  that  minimize  the  risks  and  severity  of 
danger  to  soldiers  and  the  mission  (8).  A  hazard  is  defined  as  an  actual  or  potential  con¬ 
dition  that  can  cause  injury,  illness,  or  death  of  personnel,  loss  of  equipment,  property  or 
mission  degradation.  Severity  measures  the  expected  consequence  of  an  event  (hazardous 
incident)  in  terms  of  degree  of  injury,  property  damage,  or  other  mission  impairing  factors 
that  could  occur.  Risk  is  defined  as  the  chance  of  a  hazard  or  negative  consequence;  risk 
level  is  expressed  in  terms  of  hazard  probability  and  severity  (8:G-3).  ASIST’s  hazard 
taxonomy,  a  breakdown  of  what  hazards  played  a  part  in  each  recorded  accident,  improved 
the  accident  database  by  clarifying  hazards,  controls  and  identifying  their  interactions. 

Accepted  as  the  model  that  captured  decision-makers’  values,  the  value  model  quan¬ 
tifies  the  severity  of  losses  resulting  from  accidents  for  100,000  flying  hours  for  any  given 
set  of  accident  cases.  Unlike  traditional  value-focused  thinking  where  0  is  the  least  de¬ 
sirable  and  1  is  highly  desirable,  the  model  quantifies  severity ,  therefore,  a  reduction  of 
severity  is  always  preferred.  The  Severity  of  Losses  values  are  calculated  from  accident 
data  on  the  three  most  severe  classes  of  accidents:  class  A,  any  fatalities  or  cost  over  $1 
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million;  class  B,  severe  injuries  or  cost  between  $200,000  and  $1  million;  and  class  C,  mi¬ 
nor  injuries  or  cost  between  $10,000  and  $200,000.  A  Monte  Carlo-based  bootstrapping 
simulation  returns  expected  levels  of  severity  reduction  for  blocks  of  100,000  flight  hours. 
The  Severity  Reduction  for  a  given  control  is  equal  to  the  percent  difference  between  the 
expected  Severity  of  Losses  with  no  controls  applied  and  the  expected  Severity  of  Losses 
with  the  selected  controls.  The  portfolio  optimization  uses  these  effectiveness  levels  and 
the  estimated  cost  for  each  control  to  indicate  relative  performance. 

1.2  Problem  Statement  &  Methodology 

This  research  expands  the  Severity  of  Losses  model  to  include  all  ASIST  documented 
aircraft,  proposes  methodology  to  incorporate  the  remaining  hazards,  and  formulates  an 
optimization  tool  suited  for  portfolio  selection  of  controls.  Hazards  are  system  inadequacies 
that  contribute  to  an  accident  occurring  or  its  severity.  Controls  are  measures  that  counter 
hazards  and  reduce  the  occurrence  or  severity  of  accidents.  The  focus  of  this  research  is 
on  presenting  a  methodology  and  a  working  prototype  to  integrate  multiple  systems;  the 
USASC  leadership  can  benefit  from  the  latter  insights  by  implementing  these  methods  in 
all  aspects  of  safety  that  are  categorized  into  controls  and  hazards.  Figure  1.1  shows  how 
a  single  airframe  fits  into  the  hierarchy  of  loss  contribution.  A  UH-60  accident  contributes 
to  the  overall  accident  rate  in  rotary  wing  aircraft,  in  Army  aviation,  and  the  Army  as  a 
whole.  The  goal  is  to  reduce  accident  severity  at  the  Army  level. 

The  primary  aspects  of  the  research  are  developed  separately:  the  multiattribute 
value  hierarchy,  the  bootstrap  statistics  using  Monte  Carlo  simulation,  and  the  portfolio 
optimization  model.  The  multiattribute  value  hierarchy,  based  on  the  Army’s  Risk  Man¬ 
agement  field  manual,  is  modified  using  Value-Focused  Thinking  methods  first  proposed 
by  Keeney  and  expanded  by  Kirkwood.  As  a  solid  framework  has  already  been  established 
by  Sperling  and  Gallan,  this  expanded  hierarchy  focuses  on  assessing  Army  level  versus 
battalion  level  values  and  their  weights.  This  new  Severity  of  Losses  model  integrates  ad¬ 
ditional  aircraft  as  well  as  identifies  and  analyzes  hazards  consistently,  providing  necessary 
insights  to  senior  leadership.  The  results  influence  the  selection  of  portfolios  of  controls 
that  provide  the  most  reduction  in  losses  for  given  budget  levels. 
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Figure  1.1  Scope  of  the  Study 


Assuming  the  validity  of  the  Severity  of  Losses  model  developed  by  Gallan,  the  first 
step  in  developing  a  robust,  expanded  model  was  to  integrate  the  new  accident  data  into 
a  single  data  set.  Developing  the  Simulation  to  account  for  multiple  systems,  in  this  case 
airframes,  involved  updating  measure  ranges  to  include  new  observations.  This  study 
considers  the  intent  and  expected  use  of  this  tool  by  decision  makers  when  basing  all 
severity  scores  on  the  preferences  of  experts  and  leadership. 

Once  the  model  was  developed  to  produce  an  expected  reduction  in  severity  for  any 
given  trial  portfolio,  the  search  for  a  best  portfolio  began.  Portfolio  selection  may  be 
approached  many  different  ways.  Traditionally,  decision  analysts  approach  portfolio  se¬ 
lection,  also  known  as  resource  allocation,  by  ratio  methods.  Another  reasonable  approach 
to  this  mathematical  program  was  found  in  the  knapsack  problem,  which  also  accounts 
for  resources  (22).  Exploration  of  derivative-free  optimization  methods  fall  into  a  cate¬ 
gory  called  direct  search  methods.  These  interesting  and  applicable  algorithms  efficiently 
search  regions  proceeding  to  new  points  based  upon  improvement  in  an  observed  objective 
value  (32). 

Focusing  on  the  Army’s  goal,  each  portfolio  may  contain  a  control  from  any  system 
under  study.  If  a  specific  portfolio  can  eliminate  all  of  the  accidents  for  one  aircraft, 
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the  Army’s  overall  accident  rate  may  drop  only  slightly.  The  Army  desires  to  reduce  the 
accident  severity  across  the  board,  not  just  for  one  aircraft.  Consequently,  all  aircraft  cases 
must  be  aggregated  and  analyzed,  their  hazards  noted,  and  controls  selected  to  achieve 
a  significant  drop  in  losses,  and  ultimately,  the  overall  Army  accident  severity.  The 
integrated  methodology  addressed  in  this  study  provides  an  acceptable  level  of  reduction 
in  losses  for  the  Army  while  it  remains  within  a  designated  budget  limitation. 

The  data  used  comes  directly  from  the  ASIST  database  managed  by  the  U.S.  Army 
Safety  Center,  Fort  Rucker,  AL.  ASIST,  the  Aviation  Safety  Investment  Strategy  Team, 
developed  the  hazard  taxonomy  and  controls.  This  research  takes  and  develops  an  op¬ 
timization  strategy  for  the  aircraft,  as  a  whole,  that  ASIST  has  analyzed  to  date.  This 
includes  the  UH-60  Black  Hawk,  AH-64  Apache,  CH-47D  Chinook,  and  OH-58D  Kiowa 
Warrior.  Due  to  the  massive  amount  of  excellent  analysis  done  by  ASIST,  all  of  the  data 
needed  for  this  research  is  available  and  has  been  provided  by  USASC/ORSA.  Incor¬ 
porating  this  data  and  implementing  the  methodology  utilizes  several  software  packages: 
Microsoft  Access,  Microsoft  Excel,  Frontline  Premium  Solver  Platform,  and  Palisade’s 
©RISK.  The  methodology  integrates  these  packages  to  make  use  of  specific  capabilities  of 
each. 

1.3  An  Overview  of  Subsequent  Chapters 

This  research  combines  decision  analysis,  simulation,  and  optimization  to  yield  a  spe¬ 
cialized  methodology  providing  unique  insights  and  practical  applications.  Chapter  Two 
discusses  the  literature  pertinent  to  this  study  including  decision  analysis  methods,  Monte 
Carlo  simulation  techniques,  portfolio  selection,  integer  programming,  and  simulation  op¬ 
timization.  The  complete  methodology  is  contained  in  Chapter  Three;  it  implements  the 
data  on  the  multiple  airframes  and  searches  for  the  best  portfolios  of  controls  to  meet  the 
goal  of  50%  reduction  in  severity.  Chapter  Four  discusses  the  results  of  applying  the  ASIST 
data  to  the  methodology  and  also  provides  insight  into  the  interactions  driving  portfolio 
selection.  In  addition,  sensitivity  analysis  of  the  results  provides  significant  insights  to 
whether  high  levels  of  accident  reduction  are  possible,  reasonable,  or  affordable.  Final  rec¬ 
ommendations,  insights  and  conclusions  appear  in  Chapter  Five;  this  discussion  suggests 
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improvements  to  the  data,  and  recommends  guidelines  for  selecting  tractable  portfolios. 
This  thesis  effort  concludes  with  a  discussion  of  recommendations  to  be  made  for  the  Army, 
the  Air  Force,  and  the  FAA  with  regards  to  this  study. 


1-7 


II.  Literature  Review 


2.1  Introduction 

This  chapter  first  introduces  the  reader  to  decision  analysis  and  optimization  tech¬ 
niques.  The  discussion  on  decision  analysis  addresses  Value- Focused  Thinking.  A  Monte 
Carlo  simulation  application  of  the  bootstrap  method  is  explained.  Finally,  an  introduc¬ 
tion  to  mathematical  programming  follows,  discussing  integer  programming,  the  knapsack 
heuristic,  and  direct  search  methods,  showing  how  they  can  be  combined  with  the  simula¬ 
tion  model. 

2.2  Decision  Analysis 

2.2.1  Foundational  Concepts.  Decision  analysis  is  a  prescriptive  approach  to 
help  rational  people  make  difficult  decisions  (4:3).  This  methodology  helps  the  decision 
maker  to  model  the  decision  area  and  to  integrate  the  structure  of  the  problem  with  their 
preferences  and  beliefs.  When  facing  a  decision,  every  decision  maker  tries  to  choose  the 
alternative  that  will  achieve  the  best  outcome.  An  outcome  is  the  product  of  the  decision 
made  and  the  chance  involved  given  an  uncertain  payoff,  or  future  outcome.  Because 
the  uncertainty  cannot  be  controlled,  rigorous  calculations  and  analyses  are  conducted  to 
submit  a  thorough  foundation  for  making  a  good  decision.  In-depth  studies  may  seem 
unnecessary  for  some  problems;  easy  decisions  need  not  be  subject  to  this  rigorous  analysis. 
The  intent  of  decision  analysis  techniques  is  to  provide  a  framework  for  approaching  hard 
decisions  (4:3). 

Decision  analysis  fundamentally  aids  a  decision-maker  by  providing  insight  into  the 
uncertainties,  trade-offs,  and  objectives  of  a  problem  (4:4).  At  most,  decision  analysis 
provides  a  recommended  course  of  action,  but,  in  no  way  takes  the  authority  of  decision 
making  from  the  leadership.  It  does  not  compete  with  a  decision-maker’s  intuition  or  take 
away  their  obligation  to  decide,  rather,  it  is  simply  a  valuable  tool  to  help  them  understand 
their  problem  more  clearly  and  make  a  better  decision  (4:4). 

An  analysis  of  this  type  is  not  purely  objective.  Decision  analysis  requires  personal 
input  from  experts.  These  subjective  judgements  are  a  key  ingredient  for  making  good 
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decisions.  Understanding  human  inconsistencies  is  necessary  to  carefully  apply  good 
techniques  and  recommend  improved  decisions  (4:5). 

For  a  decision-maker  to  stand  behind  a  decision  of  theirs,  they  must  understand  the 
decision  analysis  process  and  trust  the  underlying  values  and  objectives  they  have  provided. 
The  first  step  in  this  process  is  to  clearly  define  the  problem.  Although  this  may  seem 
to  be  a  simple  task,  a  superficial  problem,  or  the  question  initially  asked,  may  mask  the 
fundamental  issue  and  may  make  finding  a  clear  definition  difficult.  The  following  steps 
describe  a  method  to  help  clarify  decisions  using  Value-Focused  Thinking. 

2.2.2  Value-Focused  Thinking.  Value-focused  thinking  techniques  boast  exten¬ 
sive  use  across  the  public  and  private  sectors  of  our  economy.  Focusing  attention  on  the 
important  objectives  of  a  decision  maker  and  not  the  possible  alternatives,  value-focused 
thinking  (VFT)  techniques  provide  clear  methods  to  evaluate  alternatives.  The  develop¬ 
ment  of  value  models  by  area  experts  and  knowledgeable  decision  makers  give  a  consistent 
standard  by  which  to  measure  all  alternatives  (18).  A  formal  diagram,  or  model,  repre¬ 
sents  at  different  tiers  the  values  that  are  important  to  a  decision  maker  with  regard  to 
the  particular  issue  at  hand.  Its  purpose  is  to  determine  how  to  measure  significant  issues 
and  evaluate  alternatives  with  respect  to  the  “important  things”  (19:11). 

The  multiple  objective  value  model  (as  in  Figure  2.1)  consists  of  a  fundamental  ob¬ 
jective,  evaluation  considerations,  objectives,  and  evaluation  measures.  The  fundamental 
objective  simply  describes  the  overarching  goal.  An  example  may  be  selecting  a  new  car. 
The  evaluation  measures,  or  criteria,  are  the  important  factors  that  explain  what  is  meant 
specifically  by  the  fundamental  objective.  For  this  example,  the  evaluation  measures  could 
be  speed,  safety,  and  style.  Objectives  specify  a  preference  for  the  evaluation  consider¬ 
ations.  A  faster,  safer,  more  sleek  design  is  most  desirable  to  some,  while  a  moderate 
speed  vehicle  that  is  very  safe  and  slightly  “boxy”  might  be  the  most  desirable  to  others. 
As  is  obvious  with  the  example,  the  objectives  are  dependent  upon  the  decision  maker. 
Sub-objectives  will  likely  exist  for  more  detailed  models.  Evaluation  measures  constitute 
the  lowest  tier  of  the  value  model.  These  measures  scale  the  degree  to  which  an  alternative 
attains  desirable  objectives  (19:12).  When  navigating  through  the  tiers,  moving  to  a  lower 
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Figure  2.1  Multiple-Objective  Value  Model 


tier  requires  answering  “why  is  this  (the  current  objective)  important?”  Moving  to  higher 
tiers  answers  the  question  “what  do  you  mean  by  that  (the  current  lower  measure)?”  (27). 

The  Value  Model  consists  of  a  qualitative  part  and  a  quantitative  part.  The  quali¬ 
tative  part  is  the  value  hierarchy,  similar  to  the  general  picture  in  Figure  2.1.  Information 
that  makes  up  the  values,  objectives,  and  measures  comes  directly  from  field  experts,  doc¬ 
uments  and  doctrine,  shareholders  in  the  decision  process,  and  any  individuals  knowledge¬ 
able  about  and  interested  in  the  decision.  The  quantitative  part  consists  of  the  scoring 
functions  used  at  the  evaluation  measure  tier  and  the  weights  specified  for  all  tiers  the 
model. 

Value  hierarchies  must  meet  certain  criteria:  they  must  be  complete ,  nonredundant , 
and  independent.  The  first  two  properties  are  often  referred  to  as  mutually  exclusive  and 
collectively  exhaustive.  When  two  objectives  are  mutually  exclusive,  they  address  unique 
items  of  importance,  without  overlap.  Likewise,  a  set  of  collectively  exhaustive  objectives 
ensure  coverage  of  all  relevant  issues  of  importance.  The  evaluation  measures  must  adhere 
to  the  property  of  independence.  This  means  that  the  value  of  a  certain  level  of  one 
measure  must  not  be  dependent  on  the  level  of  a  different  evaluation  measure  (27:2.6). 
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Each  evaluation  measure  uses  data  to  evaluate  how  well  each  alternative  performs 
with  respect  to  the  objective;  the  analyst  must  decide,  with  the  decision  makers,  what  data 
is  most  representative  of  the  objectives  being  attained.  Scales  for  evaluation  measures  are 
categorized  as  natural  or  constructed  to  identify  how  clearly  the  scale  may  be  interpreted, 
and  the  measures  are  also  categorized  as  direct  or  proxy  to  identify  how  the  scale  represents 
the  objective.  Identifying  types  of  scoring  functions  enables  decision  makers  and  analysts 
to  simply  explain  how  the  data  represents  the  objective.  The  following  specific  descriptions 
and  examples  illustrate  the  most  desirable  types  of  scales  for  evaluation  measures  (19:24). 

A  natural  scale  is  commonly  used  and  is  easily  interpreted  by  most  everyone.  Profit 
in  dollars  is  a  natural  scale  that  may  be  used  to  evaluate  a  business  decision.  A  constructed 
scale  is  often  developed  with  a  certain  decision  in  mind;  often  characterized  by  categories, 
a  constructed  scale  measures  the  attainment  of  a  particular  alternative.  This  type  of 
scale  may  be  appropriate  where  a  natural  scale  is  infeasible.  The  performance  evaluation 
categories  of  a  selected  group  are  constructed  scales  (19:24)(27:2.25). 

The  direct  scale  measures  the  degree  of  attainment  of  the  objective.  A  proxy  scale 
measures  the  degree  of  attainment  of  an  associated  objective.  For  example,  “probability 
of  kill”  directly  evaluates  an  objective  Kill  Likelihood.  Whereas,  “student  grades”  are  a 
proxy  measure  for  the  attainment  of  the  associated  objective  of  student  learning  (19:24) 
(27:2.25). 


Table  2.1  Classification  of  evaluation  measures 


Natural  Constructed 

Direct 

Proxy 

1  2 

3  4 

These  combinations  of  classification  methods  for  the  evaluation  measures  create  four 
possible  combinations  of  scale  types,  and  as  the  analyst  seeks  obtain  the  most  accurate 
and  representative  hard  data  available,  the  categories  are  ranked  by  preference  in  Table 
2.1.  Natural-proxy  scales,  preference  3,  and  natural-direct  scales,  preference  1,  have  the 
benefit  of  a  pre-established  axis  for  measurement.  Any  constructed  scale  takes  significant 
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time  to  establish  and  verify.  Selection  of  appropriate  scales  always  depends  on  the  most 
fitting  measure,  however,  it  may  also  depend  on  the  data  available  to  the  decision  maker. 

After  identifying  the  scale,  a  value  function  quantifies  each  observation’s  attainment 
of  the  objective  on  a  0-1  scale.  The  most  desirable  level  receives  a  value  of  1.0  and  the 
least  desirable  level  receives  a  0.0  value.  The  intermediate  values  are  determined  by  asking 
the  preferences  of  the  decision  maker  and  obtaining  uncertainties  inherent  in  the  measure. 
Every  evaluation  measure  has  an  important  part  in  the  final  evaluation  of  the  fundamental 
objective  and  each  step  of  the  process  must  be  carefully  evaluated  and  documented  to 
ensure  future  users  are  able  to  unambiguously  assign  consistent  values  (19:28). 

The  final  step  in  developing  a  value  hierarchy  requires  quantifying  the  relative  impor¬ 
tance  of  objectives  occupying  the  same  tier.  Experts  recommend  ranking  and  comparing 
the  extremes  of  evaluation  measure  scales  using  swing  weighting  (19:64).  This  method 
consistently  evaluates  the  relative  importance  between  increments  in  value,  assuming  some 
preference  between  objectives.  Finally,  once  alternatives  are  scored  with  regard  to  the  fun¬ 
damental  objective,  varying  the  weights  reveal  the  sensitivity  and  provide  insight  about 
the  alternatives. 

2.3  Simulation  Techniques 

2.3.1  The  Bootstrap  Method.  First  published  by  Efron  in  1979,  the  bootstrap 
method  is  a  powerful  non-parametric  method  for  obtaining  an  estimator’s  accuracy  when 
data  is  limited,  specifically  providing  reliable  measures  of  uncertainty  such  as  standard 
errors  and  confidence  intervals.  Bootstrapping  empirically  generates  a  statistic’s  distribu¬ 
tion.  Using  a  Monte-Carlo  resampling  simulation,  the  bootstrapping  technique  statistically 
samples  and  analyzes  data  from  an  unknown  distribution,  generating  the  statistic’s  distri¬ 
bution  (36).  Standard  error  is  the  most  common  measure  of  an  estimator’s  accuracy;  this 
measure  is  often  difficult  to  obtain  due  to  unobservable  samples.  A  sound  approximation 
of  the  numerical  value  of  sep{6*)  is  computationally  possible  using  the  bootstrap  method 
(11:47).  The  name  for  the  bootstrap  method  originates  from  the  fictional  Baron  Mun¬ 
chausen,  who,  after  falling  to  the  bottom  of  a  lake,  pulled  himself  out  by  his  bootstraps. 
The  name  helps  explain  the  technique;  sampling  from  the  collected  data  and  evaluating  the 
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resulting  data  sets  reduces  the  sampling  error  (5:3).  As  with  any  estimation,  the  standard 
error  must  be  calculated,  and  the  smallest  error  possible  is  desired. 

To  gather  important  statistics  from  empirical  data,  the  bootstrap  method  draws  ran¬ 
dom  samples,  with  replacement,  to  generate  those  statistics.  This  method  is  particularly 
useful  in  situations  where  no  well-defined  probability  distribution  is  available.  The  pri¬ 
mary  advantages  of  using  this  method  include  its  simplicity  of  explanation  and  calculation, 
and  the  absence  of  distribution  assumptions  (11:160).  The  most  significant  drawing  factor 
for  this  research  is  that  for  complex  situations,  as  with  computationally  difficult  statistics, 
Monte  Carlo  samples  may  be  used  to  approximate  the  parameters  of  interest,  namely  the 
mean  and  its  corresponding  confidence  interval  (11:1). 

While  only  a  small  number  of  bootstrap  replications  may  be  required  to  estimate 
the  standard  error,  an  additional  factor  of  ten  replications  is  necessary  to  calculate  the 
bootstrap  confidence  interval  (11:15,  52).  Since  the  re-sampling  is  from  from  limited 
data,  the  key  sensitivities  of  the  bootstrap  method  are  similar  to  all  statistical  analyses; 
particular  attention  must  be  paid  to  confirm  correct  experimental  design,  data  analysis, 
and  especially  correct  presentation  of  conclusions  (5:4). 

2.3.2  Monte  Carlo  Simulation.  Monte  Carlo  simulation  generates  the  input 
statistics  in  a  way  that  is  analogous  to  actual  observations.  The  Monte  Carlo  samples  from 
this  simulation  are  used  for  the  bootstrap  data  samples.  Often  this  ability  to  systematically 
vary  inputs  provides  valuable  information  that  observations  cannot  supply.  The  goal  of 
Monte  Carlo  simulation  is  to  exploit  theoretical  mathematics  while  avoiding  the  risks  of 
large  variance  by  replacing  theory  with  actual  experimental  data  where  applicable  (15:4). 
Random  input  selections,  when  carefully  managed,  can  aid  in  decreasing  the  associated 
variance  within  the  system.  One  approach  is  to  use  real  observations  and  data  wherever 
practical.  However,  this  introduces  another  limitation:  the  study  infers  that  all  possible 
occurrences  are  in  the  database,  a  fact  often  known  to  be  incorrect.  The  output  statistics 
associated  with  this  simulation,  as  well  as  any,  are  only  as  accurate  as  the  observational 
data  upon  which  it  is  based  (15:4). 
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The  expected  outcome  of  a  value  model  that  contains  uncertainty  can  be  estimated 
using  Monte  Carlo  simulation.  Random  number  generation  is  the  enabler  of  this  tool. 
Every  uncertainty  that  influences  the  system  has  an  underlying  probability  distribution 
function.  By  drawing  random  numbers  from  these  distributions,  the  data  manipulation 
within  the  program  generates  a  single- value  outcome.  Repeating  this  trial  n  independent 
times  gives  reliable  data  from  which  to  statistically  analyze  and  draw  conclusions  (4). 

Currently,  many  tools  exist  to  implement  Monte  Carlo  simulation;  the  application 
of  choice  for  this  study  is  ©RISK,  add-in  software  by  Palisade  for  Microsoft  Excel.  This 
package  allows  the  user  to  uniquely  model  any  system  with  built-in  probability  distribu¬ 
tions.  As  a  simulation  progresses,  each  iteration  generates  new  possible  outcomes.  ©RISK 
keeps  track  of  the  output  values  and  generates  a  distribution  of  possible  outcomes  from 
all  the  observations.  This  distribution  is  created  by  taking  all  the  possible  output  values, 
analyzing  them  and  calculating  statistics  on  their  minimum-maximum  range  (25:51).  It 
reports  the  summary  statistics  upon  termination  of  the  experiment.  Microsoft  Excel  and 
Palisade’s  ©RISK  enable  the  Severity  of  Losses  calculations  and  projections  in  this  project. 

2.3.3  Probability  Theory .  Useful  simulations  and  statistical  inferences  rely  on 

correct  inputs.  Monte  Carlo  simulation  uses  uniformly  distributed  random  variables  as 
the  foundation  for  its  application.  In  addition,  a  specific  input  may  be  based  on  em¬ 
pirical  data,  entering  the  system  in  the  form  of  random  number  variates  from  a  specified 
distribution.  To  return  an  integer  value,  the  set  of  possible  distributions  becomes  signifi¬ 
cantly  smaller.  Often  in  practice,  interarrival  rates  are  represented  using  an  exponential 
distribution  (20:390). 

Poisson  processes  maintain  properties  that  account  for  random  occurrences  over  time 
(20:390).  Because  it  can  be  shown  that  interarrival,  or  inter-occurrence,  rates  of  accidents 
follow  an  exponential  distribution,  the  next  appropriate  step  in  modeling  accidents  for  a 
given  time  period  is  to  use  a  Poisson  process.  A  stochastic  process  {N(t),  t  >  0)}  may  be 
labeled  a  Poisson  process  if  it  adheres  to  the  following  properties  (11:160): 

1.  Events  occur  one  at  a  time. 
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2.  The  number  of  events  occurring  in  any  interval  (£,  t+s),  is  independent  of  the  number 
of  events  occurring  before  time  t. 

3.  The  distribution  of  the  number  of  events  occurring  in  any  interval  of  length  s  is 
independent  of  what  time  t  the  interval  begins. 

An  assessment  of  how  accident  occurrences  meet  these  properties  appears  in  Chapter  Three. 

2.3.4  Applications  of  Simulation  Techniques  in  Current  Literature.  Utilizing 
simulation  in  conjunction  with  the  bootstrap  method  is  explored  in  work  by  Hurry,  in  an 
application  determining  the  impact  of  Programmed  Depot  Maintenance  on  weapon  system 
availability.  His  work  supports  independent  sampling,  with  replacement,  from  empirical 
distributions.  This  analysis  produced  estimates  on  time  to  failure  and  downtime  following 
a  failure;  the  results  of  this  study  confirmed  conclusions  from  previous  analyses  (17). 

2.4  Optimization  Techniques 

Although  there  are  accepted  portfolio  selection  heuristics  within  the  decision  analysis 
community,  the  most  robust  methods  recommend  the  use  of  0-1  linear  programming  for 
finding  portfolios  with  the  highest  value  (19:207).  These  resource  allocation  methods  use 
ratios,  cumulative  cost,  or  cumulative  benefit  to  select  variables.  The  available  heuristics 
provide  acceptable,  reasonable  solutions  for  extremely  small  sets,  or  large  sets  where  preci¬ 
sion  is  not  necessary.  With  the  expanding  capabilities  of  hardware  and  software,  however, 
larger  and  more  complicated  integer  programs  are  now  significantly  easier  to  solve  than 
when  these  methods  were  first  discovered  and  used. 

There  are  several  types  of  formulation  methods  that  utilize  integer  programming 
techniques.  Seeking  to  find  an  optimal  solution,  simplicity  drives  the  program  formulation. 
If  the  problem  proves  to  be  NP-hard  with  intractable  solutions,  then  appropriate  heuristics 
may  be  used  to  approximately  solve  the  problem.  This  section  addresses  the  pertinent 
literature  describing  techniques  used  to  optimize  problems  with  characteristics  similar  to 
the  portfolio  selection  problem  addressed  in  this  study. 
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24-1  Knapsack  Problems.  This  type  of  problem  is  of  particular  interest  because  it 
suggests  attaining  the  most  benefit  possible  by  selecting  items  that,  given  a  limited  amount 
of  space,  each  contribute  a  certain  amount  of  benefit.  Similar  to  a  hiker’s  knapsack,  the 
problem  name  suggests  getting  the  most  benefit  for  the  specified  amount  of  weight  or  space 
a  knapsack  can  hold.  Often,  this  type  of  problem  allocates  a  budget,  as  the  knapsack, 
and  accepts  as  many  projects  as  possible  to  maximize  benefits.  The  general  mathematical 
definition  of  the  knapsack  problem  is: 

Maximize 


z  =  {bx|Wx  <  c,x  e  (0, 1)} 

where  bj  =  benefit  from  item  j;  W  =  weights  of  items;  and  c  is  the  cost  of  the  maximum 
size  of  items  allowed,  such  as  a  budget.  The  decision  variables,  xi ,  are  at  least  integer 
but  most  likely  binary,  indicating  that  each  item  will  either  be  included  in  the  knapsack, 
Xi  =  1,  or  left  out  of  the  knapsack,  xi  =  0.  The  classic  knapsack  problem  is  characterized 
by  the  single  constraint  that  often  acts  as  a  capacity  constraint  (maximization  problem) 
or  a  lower  bound  on  benefit  (minimization  problem). 

The  knapsack  problem  is  a  traditional  method  of  exact  algorithms  to  small  scale 
portfolio  optimization,  or  resource  allocation,  problems  (22:14).  There  are  many  methods 
to  solve  this  type  of  program;  however,  this  research  focuses  on  the  formulation  guidance 
from  this  problem,  not  its  specific  solution  methods. 

24.2  Direct  Search  Methods.  When  seeking  to  optimize  a  system,  one  must  be 
aware  of  system  characteristics,  such  as  non-differentiability,  that  may  limit  the  practical 
applications  of  common  optimization  tools.  When  designed  experiments  may  be  per¬ 
formed,  but  the  function  is  not  differentiable,  direct  search  methods  provide  convergent 
optimal,  to  within  a  pre-set  tolerance,  solutions.  Hooke  and  Jeeves  first  proposed  this  so¬ 
lution  technique  in  1961  defining  it  as  a  “sequential  examination  of  trial  solutions  involving 
comparison  of  each  trial  solution  with  the  ‘best’  obtained  up  to  that  time  together  with  a 
strategy  for  determining  what  the  next  trial  solution  will  be”  (16:212).  The  distinguishing 
characteristics  of  direct  search  include  the  ability  to  address  problems  unsuccessfully  solved 
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by  classical  methods,  a  reduced  solution  time  for  problems  able  to  be  solved  by  classical 
methods,  an  algorithm  ideally  applied  on  a  computer,  a  continually  improving  solution  set, 
and  an  ability  to  apply  different  assumptions.  Simply  described,  direct  search  selects  a 
base  point  and  a  second  point,  these  are  compared,  the  best  selected,  and  a  new  basis  is 
selected  to  obtain  the  next  value  for  evaluation  (26). 

Why  use  direct  search  methods  when  more  modern  methods  are  now  available? 
There  are  unique  features  of  direct  search  that  elude  the  problem  of  more  sophisticated 
methods  (21:2). 

2.4.2. 1  Pattern  Search.  Pattern  search  is  a  specific  strategy  for  the  direct 
search  method  that  is  characterized  by  its  particular  search  strategy.  There  are  many 
pattern  search  methods  currently  being  used  and  researched.  Most  pattern  search  algo¬ 
rithms  work  on  systems  with  continuous  variables.  Often  when  using  pattern  search  for 
R&D  designs,  the  variables  are  discrete.  The  pattern  search  strategy  is  accomplished  by 
systematically  evaluating  local  ranges  of  the  variables  and  either  accepting  a  new  solution 
based  on  a  better  result,  or  rejecting  the  solution  based  on  its  inferior  solution. 

24.2.2  General  Pattern  Search  Algorithm.  The  general  algorithm  for  pat¬ 
tern  search  consists  of  the  variables  Xi ,  incremental  step  sizes  e  for  each  x^  and  an  assump¬ 
tion  that  past  successful  strategies  will  be  successful  in  the  future.  It  is  this  assumption 
that  drives  the  two  types  of  moves,  exploratory  moves  and  search  moves.  The  procedure 
is  as  follows  (14:113).  Begin  computation  by  selecting  an  initial,  feasible  exploratory 
point;  the  method  by  which  this  point  is  selected  may  provide  a  quicker  solution,  but  any 
feasible  point  will  do.  Evaluate  the  objective  function  at  this  point  X$.  Next  evaluate 
Xi  ±  e  to  obtain  the  an  objective  value  for  y(X%).  If  there  is  improvement,  that  is,  if 
y(x  1  ±€,£2,23,  ...arn)  >  y(x ux2,x3,  —xn)  for  a  maximization  problem,  then  this  move  was 
deemed  a  success  and  the  x\  ±  e  should  be  kept  and  the  next  Xi  tried.  Sequentially,  all  the 
independent  variables  are  perturbed  with  respect  to  their  shown  improvement.  The  end 
of  this  sequence  of  moves  establishes  a  base  point ,  which  is  denoted  Xi.  Next,  providing 
that  the  exploratory  move  produced  some  improvement,  a  pattern  move  is  made  to  arrive 
at  Once  the  exploration  at  this  point  produces  X2,  an  extrapolation  of  base  points 
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Xi  and  X2  provides  X%.  This  process  of  “explore-and-extrapolate”  is  continued  until  no 
improvement  is  found.  At  that  point,  reduce  all  the  e  by  a  constant  factor  and  explore 
within  a  smaller  region  to  obtain  small  improvements.  If  success  is  found,  explore  again, 
else  reduce  e.  The  stopping  criteria  is  a  predefined  minimal  e  value  (14:114-5). 

Pattern  search  methods  were  initially  used  for  continuous  variables;  however,  this  is 
often  not  the  case  in  design  projects,  and  discrete  variables  must  be  used.  Booker  et.  al. 
discusses  a  helicopter  blade  design  in  (3:5). 

2.4.3  Knapsack  Search  Heuristic.  The  multiple  zero-one  multiple  knapsack 
heuristic  proposed  by  Pirkul  provides  a  satisfactory  method  for  solving  problems  too  large 
to  be  solved  by  optimal  solution  procedures.  As  the  knapsack  problem  has  been  used 
to  successfully  model  resource  allocation,  capital-budgeting,  and  other  decision-making 
processes,  so  this  procedure  enables  one  to  find  a  timely  solution  or  at  least  a  good  base 
solution  for  an  implicit  enumeration  method  (29:161).  Pirkul ’s  problem  involves  the  opti¬ 
mal  solutions  for  problems  with  multiple  knapsacks,  but  it  is  easily  simplified  to  a  problem 
with  one  resource  constraint.  Future  efforts  that  combine  multiple  Army  systems  should 
take  note  of  the  multiple  knapsack  application.  Noting  successful  use  of  “bang-for-buck” 
ratios,  a  ranked  list  aids  in  selecting  variables  to  set  equal  one  in  the  solution  by  simply 
evaluating  the  ratio  of  the  objective  function  coefficient  to  the  coefficient  of  the  resource 
constraint,  the  benefit-cost  ratio.  The  general  approach  selects  the  variables  with  the 
greatest  bi/ci  ratio,  or  the  greatest  likelihood  of  marginal  benefit,  to  add  to  the  objective 
function. 

The  formal  heuristic  procedure  modified  for  a  single  portfolio  is  as  follows.  First, 
calculate  the  6;/c,  ratios  and  sort  all  choices  i  in  decreasing  order.  Next,  fix  variables 
equal  to  1  based  on  order  in  the  first  step,  and  pay  particular  attention  to  not  violating 
constraints.  The  base  feasible  solution  is  denoted  x.  Finally,  for  each  variable  that  is 
equal  to  1  in  x,  fix  that  variable  to  1  and  repeat  the  previous  step  to  use  the  remainder  of 
the  resources  and  generate  a  new  trial  solution.  For  each  base  solution,  there  will  be  an 
associated  number  of  trial  solutions  that  equal  the  number  of  variables  selected  in  the  base 
solution.  Following  a  single  base  solution,  the  best  trial  solution  becomes  the  next  base 
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point  for  a  search.  These  iterative  search  regions  continue  until  the  predefined  minimum 
improvement  between  variables  has  been  reached  (29:165). 

This  heuristic  is  very  applicable  to  this  study  in  that  the  traditional  knapsack  problem 
could  not  be  implemented  because  of  unavailable  objective  function  coefficients.  Direct 
search  procedures  for  zero-one  problems  have  not  been  clearly  set  nor  proven  to  converge 
unlike  the  related  direct  search  integer  program  (32).  This  heuristic  allows  for  adjustment 
without  violating  assumptions  of  the  method.  It  enables  the  calculation  of  acceptable 
answers  to  problems  that  the  knapsack  problem  and  the  direct  search  procedure  pose.  In 
the  future,  development  of  a  direct  search  procedure  for  zero-one  problems  will  likely  be 
published,  and  this  technique  should  be  first  examined. 

2.4.4  Optimization  Summary .  The  technique  for  systematically  examining  port¬ 
folios  integrates  the  output  of  the  simulation  model  and  the  modified  direct  search  tech¬ 
niques  to  recommend  regions  of  acceptable  response.  Operationally,  the  Army  addressed 
these  issues  using  intuition  and  qualitative  assessments.  This  methodology  allows  the  de¬ 
cision  maker  to  make  qualitative  assessments  from  the  resulting  quantitatively  developed 
portfolios. 
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III.  Methodology 


3. 1  Overview 

This  chapter  discusses  the  methodology  that  was  used  to  select  the  optimal  set  of 
controls  to  reduce  the  cost  of  accidents,  the  number  of  casualties,  and  the  accident  rate. 
The  analysis  can  be  divided  into  two  sections:  first,  the  Severity  Reduction  simulation 
which  includes  the  Severity  of  Losses  value  model  and  second,  the  mathematical  portfolio 
optimization  program. 

3. 2  An  explanation  of  the  Severity  Reduction  Simulation. 

The  purpose  of  the  severity  reduction  simulation  is  to  produce  the  expected  reduction 
in  severity  over  100,000  flight  hours  when  a  specific  portfolio  of  controls  is  applied.  One 
hundred  thousand  flight  hours  is  the  aviation  standard  measure  for  calculating  accident 
rates.  The  percent  difference  between  the  expected  severity  for  a  portfolio  with  no  controls 
applied  and  the  portfolio  with  certain  selected  controls  applied  defines  the  reduction  in 
severity.  A  single  iteration  of  the  simulation  calculates  the  severity  of  the  accidents 
selected  for  100,000  flight  hours  as  a  specific  portfolio  is  applied.  The  expected  severity 
is  obtained  by  completing  multiple  iterations,  each  calculating  the  severity  for  the  given 
portfolio. 

Monte  Carlo  simulation  techniques  are  used  to  generate  a  number  of  possible  acci¬ 
dents  to  select  from  the  sample  data  to  simulate  a  single  100,000  flight  hour  block.  Ref¬ 
erence  to  an  iteration  indicates  a  simulated  set  of  accidents  for  five  years  worth  of  100,000 
flight  hours  blocks  and  the  resulting  severity,  given  the  application  of  a  specific  portfolio  of 
controls.  The  next  sections  describe  necessary  elements  of  the  simulation  in  more  detail. 

3.2.1  ASIST  Database.  The  Army  Safety  Center  initiated  the  meeting  of  avia¬ 
tion  industry  and  miliary  experts  to  analyze  accident  data.  This  group  of  experts,  the 
Aviation  Safety  Investment  Strategy  Team,  ASIST,  met  to  identify  hazards  causing  in¬ 
dividual  accidents  and  address  the  possible  controls  that  may  be  applied  to  reduce  the 
likelihood  or  severity  of  the  accident.  Because  the  objective  of  the  analysis  was  to  take  a 
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risk-based  approach  rather  than  a  mistake-based  or  blame-focused  look  at  accidents,  the 
resulting  information  focuses  on  prevention  (35:2000).  Days  of  analysis  by  subject  matter 
experts  resulted  in  an  extremely  useful,  focused  set  of  accident  cases.  This  foundational 
set  of  actual  accidents  included  accident  data,  the  hazards  affecting  these  accidents,  and 
the  associated  controls  that  reduce  the  effect  of  hazards.  This  database  supports  every 
part  of  the  simulation. 

3. 2.1.1  Hazards .  ASIST  chose  to  set  their  focus  on  identifying  hazards, 

not  on  identifying  cause  factors  for  specific  accidents.  A  hazard  is  any  factor  contributing 
to  the  probability  or  severity  of  an  accident.  Defining  hazards  is  significantly  different 
from  identifying  causes.  The  distinction  involves  the  focus  of  the  analysis:  hazards  are 
prevention  focused,  while  causes  are  blame  focused.  Analysis  often  emphasizes  mistakes 
when  identifying  the  causes;  however,  looking  to  hazards  tends  to  create  a  concentration 
on  risk.  The  taxonomy  focuses  attention  on  the  parts  of  the  man-machine-environment 
system  which  together  produce  hazards  (13:50).  The  Army  identified  and  published  tech¬ 
niques  for  Risk  Management  most  recently  in  Apr  1998,  through  FM  100-14.  The  haz¬ 
ards  perspective  provides  the  Army  with  methods  of  protecting  of  Army- wide  investments 
(35:2000). 

Building  a  hazard  taxonomy  to  recognize  risk  involves  viewing  the  accident  cases  with 
a  new  mindset,  one  that  sees  the  mission,  leadership,  aircraft,  and  crew  as  contributors  to 
the  case  as  a  whole.  Combinations  of  all  these  parts  may  provide  the  most  comprehensive 
determination  of  risk  involved  in  the  accidents  as  well  as  the  degree  to  which  soldiers  are 
exposed  to  the  risk  (35:2000). 

In  addition  to  the  development  of  the  hazard  taxonomy,  ASIST  assessed  the  hazard 
contribution  for  casualty,  cost,  and  frequency.  Each  assessment  estimates  the  percent  a 
specific  hazard  contributes  to  the  accident’s  cost,  casualties,  and  occurrence.  A  hazard 
contributing  to  casualties  specifically  affects  injuries  and  fatalities.  A  hazard’s  contribution 
to  cost  indicates  the  percent  of  the  total  accident  cost  for  which  this  hazard  was  responsible. 
Finally,  assessments  of  frequency  refer  to  causes  contributing  to  the  occurrence  of  the 
accident.  An  example  case  with  contributing  hazards  follows  in  Table  3.1. 
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Table  3.1  Contributing  Hazards 


Accident 

Contributing 

Casualties 

Cost 

Frequency 

Case 

Hazards 

Contribution 

Contribution 

Contribution 

19931019001 

OH58-58 

50 

50 

AVN-02 

50 

50 

Total 

100 

100 

100 

19960116001 

AH64-62 

0 

1 

8 

AH64-69 

50 

49 

31 

AH64-70 

0 

1 

15 

AH64-71 

50 

49 

31 

AH64-85 

0 

0 

15 

Total 

100 

100 

100 

3. 2. 1.2  Controls.  As  part  of  their  analysis,  ASIST  identified  controls  which 
are  actions  taken  to  eliminate  hazards  or  reduce  their  risk  (8).  A  typical  value  model 
develops  and  evaluates  alternatives  based  on  what  is  important  to  the  decision  maker. 
For  this  study,  controls  are  the  alternatives;  they  were  generated  by  the  ASIST  team  in 
response  to  outstanding  hazards.  In  ASIST,  the  controls  were  developed  using  the  Army’s 
Doctrine,  Training,  Leader  Development,  Organization,  Material,  and  Soldier  Performance 
(DTLOMS)  framework.  Controls  may  affect  more  than  one  hazard,  each  with  a  differ- 
ent  effectiveness.  MIL-STD-882  system  safety  design  order  of  preference  determines  the 
individual  control  effectiveness  estimates: 


Table  3.2  Control  Effectiveness  Estimates 


1. 

Design  for  minimum  risk 

80-100% 

2. 

Safety  devices 

60-80% 

3. 

Warning  devices 

40-60% 

4. 

Procedures  and  training 

20-40 

The  individual  control  effectiveness  estimates  for  each  hazard  indicate  the  assessed 
percent  reduction.  The  area  experts,  Army  aviators  and  other  members  of  ASIST,  dedi¬ 
cated  many  long  hours  to  estimating  hazard  reduction  achieved  at  the  end  of  ten  years  of 
control  application.  ASIST  has  proposed,  to  date,  353  controls  for  the  force-modernized 
aircraft  addressed  in  this  study. 
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Control  cost  is  categorized  by  the  range  of  possible  expenditures  for  20-year  imple¬ 
mentation  fleetwide.  The  optimization  uses  the  median  value  of  the  probable  range  to 
estimate  cost.  Because  controls  will  be  selected  for  fleetwide  use,  individual  aircraft  cost 
is  not  considered.  Although  only  estimates,  these  costs  provide  a  resource  value  by  which 
controls  may  be  added  to  budget-specific  portfolios. 


Table  3.3  Estimated  Control  Cost  Ranges 


20  year  fleetwide  cost 

Point  Est.  Cost 

$  0  -  100K 

nil 

$  50,000 

$  100K  -  1M 

very  low 

$  500,000 

$  1M  -  38M 

low 

$  19,000,000 

$  38M  -  150M 

medium 

$  90,000,000 

>  $  150M 

high 

$  200,000,000 

3.2.2  Possible  Accident  Input  In  the  simulation,  the  possible  accidents  for  each 
iteration  are  randomly  drawn  from  past  accident  reports.  Replacement  of  the  accident 
data  allows  the  possibility  of  the  same  accident  occurring  more  than  once  within  a  single 
iteration.  Due  to  the  aggregation  of  all  accident  data  for  the  force  modernized  aircraft, 
AH-64,  CH/MH-47,  UH/MH-60,  and  OH-58D,  the  number  of  possible  accidents  for  each 
iteration  is  based  on  their  overall  accident  rate  during  the  past  seven  years.  This  method 
of  accident  selection  permits  the  future  accidents  per  aircraft  type  to  be  based  on  the 
accident  data.  This  removes  the  need  to  aggregate  by  aircraft  type  using  normalized 
hours  flown  or  hours  per  actual  aircraft  per  month. 

By  inspecting  the  accident  cases,  it  is  noted  that  accidents  occur  independently  of 
one  another.  The  number  of  possible  incidents  in  an  interval  of  100,000  flight  hours  can  be 
represented  by  a  Poisson  random  variate  (20:390).  Each  case  represents  a  single  incident 
occurring  at  one  point  in  time,  which  meets  the  first  property  of  a  Poisson  process.  Next, 
the  occurrence  of  accidents  in  one  100,000  flight-hour  unit  is  not  dependent  on  previous 
incidents  or  on  “run-up”  time,  as  with  the  stand-up  of  a  system.  The  occurrence  of 
accidents  in  one  set  of  100,000  flight  hours  is  independent  from  accidents  occurring  in  a 
completely  separate  or  even  an  overlapping  set  of  100,000  flight  hours.  Thus,  a  Poisson 
random  variate  is  well  suited  for  generating  inputs  in  this  simulation. 
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The  most  recent  accident  occurrence  information  best  forecasts  future  accident  oc¬ 
currences.  Future  possible  accidents  are  derived  from  the  Poisson  distributed  accident 
rates  for  FY94  to  FYOO,  the  period  ASIST  has  analyzed  to  the  present.  The  actual 
number  of  possible  accidents  in  one  iteration  is  produced  by  a  random  variate  generator 
based  on  the  Poisson  distributed  accident  data;  for  this  analysis,  a  Poisson  distribution 
with  parameter  A  =  11.89,  the  mean  estimated  from  the  accident  data  available. 

3.2.3  Control  Effectiveness.  Each  selected  control  reduces  the  occurrence  or 
likelihood  of  certain  hazards  by  an  appropriate  efficiency  designated  by  the  ASIST  team. 
The  controls  do  not  reduce  all  hazards  with  the  same  efficiency.  For  this  reason,  the  lists 
of  controls  must  be  examined  to  ensure  that  every  hazard  reduced  by  the  selected  portfolio 
of  controls  is  included.  If  multiple  controls  help  reduce  the  same  hazard,  the  difference  in 
reduction  amounts  must  be  addressed.  When  ASIST  decided  these  effectiveness  measures, 
they  called  it  a  reduction  in  the  likelihood  of  an  accident.  It  is  assumed  in  the  study  that 
each  reduction  is  to  be  taken  from  the  remaining  likelihood  of  an  accident.  Consider  this 
situation:  three  different  controls  all  reduce  the  same  hazard,  but  by  different  amounts. 
If  control  H47-C05  reduces  hazard  H47-12  by  0.5;  control  H47-C16,  by  0.5;  and  control 
H47-C45,  by  0.25,  we  cannot  say  that  the  total  reduction  for  H47-12  is  1.25.  Noting  the 
difficulty  in  completely  evaluating  the  hazard  list  resulting  from  every  control  combination, 
a  generic  method  of  combining  reductions  is  suggested.  The  additive,  but  not  greater 
than  1.0  model  may  simplify  calculations,  but  such  an  approach  incorrectly  assumes  that 
hazards  may  be  completely  eliminated.  A  more  conservative,  and  more  realistic  method  of 
integrating  the  hazards  is  a  multiplicative  combination,  where  a,  represents  the  individual 
effectiveness  values: 

i 

Combined _Hazard  ^Reduction  =  i-n<w 

1 

Therefore  take  25%  of  50%  of  50%;  the  result  is  a  reduction  of  0.8125.  This  is  the  hazard 
reduction  input  into  the  model. 
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3.24  Severity  of  Losses  Model  The  Army  Safety  Center  seeks  to  reduce  accident 
severity  by  50%  over  the  next  ten  years  (beginning  April  1999).  By  reducing  accident 
severity,  the  important  underlying  issues,  casualties  and  total  cost,  are  also  reduced.  The 
Severity  of  Losses  model  is  a  value-focused  thinking  model  which  addresses  the  important 
facets  of  accident  severity  and  accounts  for  the  preferences  of  commanders  to  describe  why 
severity  is  important.  For  the  purpose  of  this  analysis,  it  is  assumed  that  reducing  the 
frequency  or  severity  of  contributing  hazards  should  reduce  the  overall  losses  associated 
with  the  accidents.  The  model  describes  the  severity  of  accidents  in  100,000  flight  hours. 
Severity,  from  a  safety  standpoint,  must  be  minimized.  To  support  a  model  that  experts 
and  users  alike  understand,  a  value  of  1.0  is  given  to  the  worst  severity  score,  and  a 
severity  score  of  0.0  is  the  best,  or  least  severe.  Value-focused  thinking  models  often  scale 
all  alternatives  on  a  0  to  1  scale;  however  for  severity,  the  goal  is  to  obtain  a  score  near 
zero  (19). 

Army  FM  100-14,  entitled  “Risk  Management,”  states  four  criteria  for  assessing 
severity:  degree  of  injury  or  illness,  repair  or  replacement  costs,  other  mission  impairing 
factors,  and  environmental  damage  (8:2-9).  From  these  criteria,  the  top  tier  of  the  hier¬ 
archy  is  derived:  casualties  corresponds  to  the  degree  of  injury  or  illness,  unit  readiness 
corresponds  to  other  mission-impairing  factors,  total  costs  correspond  to  loss  of  or  damage 
to  equipment  or  property,  and  environmental  damage  remains. 


Figure  3.1  Top  Level  of  Severity  of  Losses  Hierarchy 


Each  of  these  objectives  are  supported  in  FM  100-14  as  the  key  objectives  describing 
severity.  The  following  sections  address  the  reasons  behind  the  development  of  these 
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branches.  The  severity  functions  for  every  evaluation  measure  are  completely  developed 
in  detail  in  Appendix  A. 

3,2.4. 1  Casualties.  Casualties  measures  the  contribution  to  severity  made 
by  fatalities  and  injuries  sustained  due  to  accidents.  According  to  Army  Risk  Manage¬ 
ment,  FM  100-14,  and  USASC  leadership,  the  following  subobjectives  effectively  measure 
casualties  contribution  to  severity  from  the  Army-level  perspective  (Warren,  2000).  These 
components  are  shown  in  Figure  3.2  and  addressed  individually  in  the  following  subsec¬ 
tions. 


Figure  3.2  Sub-objectives  for  Casualties 


Loss  of  Life  assesses  the  impact  of  all  fatalities  on  the  Army;  it  measures  the  severity 
of  the  number  of  fatalities  for  100,000  flight  hours.  This  natural-direct  measure  includes 
all  fatalities  resulting  from  an  accident,  military  and  civilian  personnel. 

Permanent  disabilities  fall  into  two  categories,  partial  and  total.  The  Army  Safety 
center  tracks  both  types  of  injuries  in  the  Risk  Management  Information  Database  (RMIS). 
Total  disabilities ,  as  determined  by  the  opinion  of  competent  medical  authority,  compro¬ 
mise  any  nonfat al  injury  that  permanently  and  totally  incapacitates  a  person  to  the  extent 
that  he  or  she  cannot  find  any  gainful  employment  (DA  PAM  385-40,  1994:  Glossary). 
Rarely  do  these  types  of  injuries  result  from  aviation  accidents.  The  Severity  of  Losses 
model  accounts  for  this  type  of  loss;  however,  due  to  the  lack  of  any  occurrence  of  perma¬ 
nent  total  disabilities  in  this  data  sample,  this  measure  is  not  used  in  this  study. 

Permanent  partial  disabilities  is  a  natural-direct  measure,  and  it  assesses  the  contri¬ 
bution  to  severity  of  losses  from  permanent  disabilities.  Although  also  rare,  its  occurrence 
is  severe  enough  to  include  in  the  model.  A  permanent  partial  disability  is  any  injury  (not 
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resulting  in  death  or  permanent  total  disability)  that,  in  the  opinion  of  competent  medical 
authority,  results  in  the  loss  or  permanent  impairment  of  any  part  of  the  body,  with  the 
following  exceptions: 

1.  Loss  of  teeth. 

2.  Loss  of  fingernails  or  toenails. 

3.  Loss  of  tip  of  fingers  or  tip  of  toe  without  bone  involvement. 

4.  Inguinal  hernia,  if  it  is  repaired. 

5.  Disfigurement 

6.  Sprains  that  do  not  cause  permanent  limitation  of  motion 
(DA  PAM  385-40,  Glossary). 

Time  incapacitated  is  a  natural-direct  measure  of  the  severity  of  losses  resulting  from 
days  of  hospitalization  due  to  aviation  accidents.  “Admission  to  a  hospital  as  an  inpatient 
for  medical  treatment”  officially  defines  hospitialization  (DA  PAM  385-40,  1994:  Glossary). 
Days  hospitalized  counts  only  the  hospitalization  of  military  personnel. 

3. 2.4,2  Unit  Readiness.  Any  loss  not  only  affects  the  Army  as  a  whole, 
it  certainly  directly  impacts  the  individual  units  as  well.  The  unit  readiness  objective 
measures  the  severity  of  losses  in  terms  of  the  impact  these  losses  have  on  individual 
battalions.  As  representative  of  FM  100-14’s  mission-impairing  factors,  unit  readiness 
identifies  the  following  sub-objectives  as  major  contributing  factors:  training  execution , 
unit  morale ,  and  equipment  availability. 

Training  execution  is  a  natural-proxy  scale  that  assesses  the  impact  of  decreased 
training  on  unit  readiness.  Commanders  are  believed  to  become  more  risk  averse  as 
accidents  increase  within  the  unit.  The  battalion  commander  may  react  to  recent  accidents 
by  reducing  the  complexity,  realism,  sophistication  of  training,  or  amount  of  training  for 
the  unit.  This  change  in  mission  plans  and  execution  results  in  decreased  unit  readiness. 
Training  execution  measures  the  number  and  class  of  the  accidents  that  occur  in  a  single 
unit.  Assessment  and  opinions  here  are  drawn  from  an  interview  with  LTC  Semmens  (13). 
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Figure  3.3  Sub-objectives  for  Unit  Readiness 


Soldiers  within  a  unit  drive  the  level  of  readiness.  Any  decrease  in  unit  morale  will 
have  a  detrimental  effect  on  unit  readiness.  Although  many  issues  may  be  identified  to 
assess  the  impact  of  aircraft  accidents  on  unit  morale,  two  major  influences  were  identified: 
loss  of  life  and  magnitude  of  injury.  Loss  of  life  is  a  natural-proxy  measure  for  assessing 
the  decrease  in  unit  morale  due  to  the  number  of  fatalities  in  the  unit.  The  emotional 
impact  on  soldiers  affects  performance  and  consequently  unit  morale.  Acknowledging 
that  fatalities  are  not  the  only  factor  impacting  unit  morale,  magnitude  of  injury  uses 
a  constructed  scale  to  measure  how  permanent  disabilities  and  hospitalizations  of  fellow 
soldiers  result  in  decreased  morale.  This  proxy  scale  is  weighted  to  indicate  that  permanent 
injuries  have  a  large  impact,  as  do  lengthy  hospitalizations. 


Table  3.4  Injury  Category  Classification 


Category  0: 
Category  1: 
Category  2: 
Category  3: 


No  injuries  requiring  hospitalization 
Injuries  requiring  <  7  days  hospitalization 

Injuries  requiring  >  7  days  hospitalization,  no  permanent  disabilities 
Injuries  resulting  in  permanent  disabilities _ 


Equipment  availability  assesses  the  impact  on  unit  readiness  of  having  fewer  aircraft 
for  a  unit.  This  natural-proxy  measure  assesses  the  number  of  unavailable  aircraft  within 
the  unit.  An  aircraft  is  defined  unavailable  when  it  is  deemed  a  total  loss  or  requires 
greater  than  40  man-hours  to  repair.  If  an  aircraft  requires  more  than  40  man-hours  to 
repair,  it  is  assumed  that  repairs  would  not  be  accomplished  at  the  unit  level,  but  rather 
at  post  or  depot  level  (13).  An  aircraft  will  be  considered  a  total  loss  when  it  is  not 
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economically  repairable.  An  estimate  of  man-hours  required  first  appears  in  the  accident 
report;  however,  the  final  number  of  hours  necessary  are  recorded  and  this  study  uses 
this  man-hour  total.  Time  to  assess  and  estimate  damage,  as  well  as  repair  and  replace 
damaged  and  not  economically  repairable  parts,  are  all  counted  for  the  final  total  (9:2-11). 

3.24.3  Total  Costs.  Total  costs  is  a  natural-direct  measure  assessing  the 
severity  of  accidents  in  terms  of  the  dollar  cost  to  the  Army.  Army  Regulation  735-11 
includes  the  following  as  Army  accident  costs:  injury  costs,  repair/replacement  costs, 
and  other  military  and  non-military  damage  costs  resulting  from  accidents.  Accident 
investigation  teams  use  accident  cost  to  determine  the  classification. 


[ 

Total  Costs 

—  Repair/Replacement  + 
Injury  Costs  + 

Damage  Costs 

Figure  3.4  Details  of  Total  Cost  Objective 


Repair  and  replacement  costs  include  all  costs  incurred  from  aircraft  damage.  In 
the  event  of  a  total  loss,  the  acquisition  cost  is  used  as  the  cost  to  replace.  If  the  aircraft 
is  repairable,  man-hour  costs  and  parts  replacement  costs  will  be  used  (9:2-11). 

Injury  costs  include  medical  expenses;  however,  they  do  not  estimate  the  monetary 
loss  incurred  while  the  soldier  is  not  mission  capable.  Specifically,  indirect  costs  associated 
with  the  accident  such  as  production  loss,  or  wages  lost  to  employees  not  injured,  and  the 
cost  of  hiring  and  training  new  employees  are  not  included.  Injury  costs  consist  of  the 
cost  of  pay  while  away  from  work,  medical  treatment,  hospitalization,  dependent  survival, 
unused  training  costs,  gratuities,  compensation,  disability  retirement,  and  burial.  An 
actual  time  away  from  work  is  included,  if  known,  when  the  accident  report  is  submitted; 
otherwise,  an  estimate  of  lost  workdays  made  by  a  competent  medical  authority  is  used 
(9:2-11). 
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3.244  Environmental  Damage .  The  final  objective  contributing  to  the 

severity  of  losses  is  environmental  damage.  Balancing  damage  to  the  environment  against 
national  objectives,  often  means  choosing  the  national  objective,  yet  doing  everything  rea¬ 
sonable  to  minimize  environmental  damage.  The  US  Army  Engineer  School,  the  executive 
agent  for  Military  Environmental  Protection,  provides  the  applicable  sub-objectives  in  the 
Military  Environmental  Protection  Manual  (7).  Damage  is  categorized  by  the  element 
damaged;  the  Army  is  concerned  with  spills  that  will  damage  the  soil  and  water.  Air¬ 
craft  accidents  may  leak  fuel,  hydraulic  fluid  and  oil.  Currently,  a  detailed  assessment  of 
environmental  damage  is  not  available  due  to  the  lack  of  data. 


Figure  3.5  Sub-objectives  for  Environmental  Damage 


The  Army  categorizes  hazardous  fluid  spills  by  the  number  of  gallons  spilled.  Because 
of  its  importance,  the  measures  for  an  assessment  of  severity  based  on  spill  type  and 
environment  rely  on  this  limited  information.  The  constructed-direct  measure  of  severity 
of  environmental  damage  accounts  for  effects  on  environment  and  clean-up  required. 


Table  3.5  Hazardous  Fluid  Spills  Classification 


Category  0: 
Category  1: 
Category  2: 
Category  3: 
Category  4: 
Category  5: 


No  hazardous  fluid  spilled 
Less  than  1  gallon  of  hazardous  fluid  spilled 
More  than  1  gallon,  but  less  than  2  gallons 
More  than  2  gallons,  but  less  than  10  gallons 
More  than  10  gallons,  but  less  than  20  gallons 
More  than  20  gallons  spilled 
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3.2.5  Bootstrapping  Using  Monte- Carlo  Simulation.  Bootstrapping  statistics 
using  Monte-Carlo  samples  is  employed  in  this  simulation  by  using  Palisade’s  @RISK 
software.  This  technique  statistically  samples  and  analyzes  data  from  an  unknown  distri¬ 
bution.  The  number  of  possible  accidents  for  one  iteration  is  generated  by  a  pre-defined 
random  Poisson  variate.  Once  the  number  of  accidents  to  evaluate  has  been  selected,  the 
actual  accidents  to  be  simulated  are  drawn  randomly,  with  replacement,  from  the  complete 
list  of  cases  evaluated  by  ASIST.  Independence  of  runs  is  assumed. 

3. 2. 5.1  Number  of  Sample  Runs.  The  number  of  runs  in  part  determines 
the  accuracy  of  an  estimator.  The  estimator  of  interest  for  this  study  is  percent  severity 
reduction,  as  generated  by  the  Severity  Reduction  Simulation.  To  evaluate  portfolios  of 
controls  with  regard  to  one  another,  an  estimate  of  the  performance  measure  within  an 
accuracy  e  with  specified  confidence  100(1— cc)%  must  be  known.  To  determine  the  number 
of  runs  necessary  to  obtain  the  relevant  statistics,  means  and  90%  confidence  intervals,  use 
of  the  equation  for  the  student  t  statistic  allows  for  the  determination  of  the  number  of 
replications,  R,  needed.  R  replications  are  always  greater  than  R0  sample  runs  (2:439). 

R  >  (ta/2, r-i  x  S0/e)2  where  a  =  0.1,  S2  =  the  variance  for  Ra  sample  runs, 
and  tQ/2,R-iis  determined  from  the  student  t  tables.  For  this  study,  the  average  severity 
reduction  for  27  blocks  of  100,000  flight  hours,  an  estimated  five  year  period,  composes 
a  single  sample  run.  The  variance  observed  for  a  particular  run  is  S0  —  0.00033.  The 
specified  accuracy  is  e  =  0.01,  or  1%. 

Using  the  cumulative  normal  distribution, 

R  >  (za/ 2  x  S0/e)2  -  (1.6452  x  0.00033/0.012)  =  8.93. 

Since  R  must  be  >  9,  proceeding  with  the  student -t  equations,  the  smallest  integer  R 
that  satisfies  the  inequality  is  11:  11  >  (1.812  x  0.00033/0.012)  =  10.81. 

Thus,  to  estimate  the  mean,  11  replications  must  be  performed.  In  accordance 
with  Efron  and  Tibshirani,  ten  times  the  number  of  replications  should  be  performed 
to  accurately  estimate  the  confidence  intervals  using  the  bootstrap  method.  Therefore, 
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110  replications  are  run  when  estimating  confidence  intervals.  The  expected  Severity 
Reduction  for  any  one  block  of  100,000  flight  hours  in  the  estimated  five  years  of  Army 
flight  hours  is  passed  to  the  optimization  program. 

3.2.6  Process  Overview.  One  iteration  consists  of  a  portfolio  of  controls  input,  a 
possible  accident  draw,  and  a  severity  reduction  output.  First,  because  each  control  has  a 
reduction  efficiency  for  certain  hazards,  the  input  control  list  is  reviewed  and  translated  into 
terms  of  hazards  and  their  effectiveness.  Next,  the  assigned  number  of  accidents  from  the 
draw  are  selected  from  the  database  and,  where  applicable,  the  accident  data  is  reduced  as 
it  could  have  happened  when  the  control  was  applied.  The  resulting  cumulative  accident 
data  for  one  iteration  is  next  evaluated  in  the  Severity  of  Losses  value  model  to  obtain 
the  Severity  value  for  both  what  the  accidents  would  have  been  without  the  portfolio 
of  controls  and  what  the  accidents  result  is  with  the  portfolio  applied.  As  previously 
mentioned,  this  procedure  is  repeated  to  obtain  reliable  statistics  for  its  expected  value. 
A  detailed  explanation  is  included  in  Appendix  B. 


Figure  3.6  The  Severity  Reduction  Simulation  Complete  Process  Diagram 
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3.3  Optimizing  the  Portfolios 

The  Army’s  objective  is  to  obtain  a  50%  reduction  in  accident  severity,  to  gain  the 
greatest  reduction  in  accident  rate,  accident  cost,  and  casualties.  The  first  section  of  this 
chapter  established  the  model  which  returns  an  accepted  estimate  of  severity  reduction, 
an  overall  measure  of  the  Army’s  objective.  This  section  establishes  the  best  technique 
for  selecting  sets  of  controls  based  on  a  given  budget.  This  study  analyzes  a  set  of  353 
controls.  Each  control  may  be  selected,  or  may  not  be  selected  for  the  portfolio;  this  results 
in  2353,  or  1.8  x  10106  different  possible  portfolios,  clearly  computationally  intractable  to 
completely  enumerate. 

3.3.1  Unique  Problem  Characteristics.  This  portfolio  selection  problem  is  unique 
because  of  several  features:  first,  the  objective  function  is  not  differentiable;  second,  the 
interaction  between  controls  is  unpredictable;  and  third,  the  size  of  the  problem  makes 
enumeration  impossible.  There  are  procedures  in  existence  to  accommodate  some  of  these 
characteristics,  and  an  eclectic  approach  that  ensures  key  assumptions  are  met  is  the  best 
solution. 

It  is  possible  to  evaluate  controls  individually  and  then  take  a  traditional  portfolio 
optimization  approach  as  suggested  by  Kirkwood  or  Keeney;  however,  both  assume  addi¬ 
tivity  of  the  values  (19:207).  The  nature  of  the  ASIST  data  set,  as  described  in  Section 
3.2.3,  and  the  case  information  suggest  that  an  additive  severity  reduction  is  not  appro¬ 
priate.  Consequently,  the  fundamental  assumption  based  on  the  data  sets  is  that  when 
multiple  controls  are  applied,  a  synergistic  effect  may  be  observed.  By  observing  the 
expected  severity  reduction  of  a  specified  portfolio,  iterative  adjustments  provide  insight 
into  the  possible  interactions  present  returning  the  greatest  improvement. 

3.3.2  The  Search  Procedure.  Following  Pirkul’s  multiple  knapsack  heuristic, 
pertinent  observations  may  be  made  and  useful  solutions  can  be  obtained.  The  Severity 
Reduction  Simulation  takes  a  portfolio  and  accident  distribution  input  and  returns  the 
expected  severity  reduction  (%)  for  that  portfolio.  In  order  to  estimate  the  impact  each 
control  makes  when  it  is  evaluated  alone,  a  portfolio  is  created  with  only  one  control  in  it. 
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Obtaining  a  reasonable  starting  point  begins  by  evaluating  all  353  single  control  portfolios. 
Acknowledging  that  the  sum  of  these  reductions  by  themselves  exceed  1.0,  the  interactions 
must  be  both  positive  and  negative.  In  addition,  there  are  specific  constraints  inherent 
to  the  controls  that  must  be  considered.  For  example  H47-C09  and  H47-C10  would  never 
logically  be  applied  simultaneously.  To  avoid  this  procedure  recommending  infeasible 
portfolios,  multiple  choice  constraints,  as  appropriate,  are  built  into  the  base  portfolio 
optimization  procedure. 

The  program  developed  in  the  study  uses  traditional  binary  integer  programming 
techniques  to  maximize  a  surrogate  objective  function  where  the  coefficients  for  each  de¬ 
cision  variable  are  the  reduction  values  from  the  each  single  control  portfolio.  The  con¬ 
straints  consist  of  the  resource  constraint,  the  budget,  and  the  mutually  exclusive  control 
application  constraints  that  restrict  specific  combinations.  Finally  as  defined  by  the  pro¬ 
gram  type,  all  decision  variables  in  this  program  are  binary.  The  solution  obtained  by 
this  surrogate  objective  function  purely  estimates  the  magnitude  of  reduction.  The  actual 
percent  severity  reduction  returned  by  the  specified  portfolio  is  less  than  the  surrogate. 

Base  points  for  several  levels  of  representative  budgets  are  analyzed;  the  results 
provide  insights  into  what  the  expected  level  of  reduction  for  a  given  budget  could  be.  In 
each  additional  portfolio  analysis,  the  knapsack’s  resource  constraint,  in  this  case  budget, 
appears.  A  linear  comparison  of  what  the  integer  program  expects  versus  what  the  Severity 
Reduction  Simulation  actually  returns  initially  reveals  the  direction  that  the  expansion 
should  take. 

Incorporating  direct  search  derivative-free  methodology,  effective  and  efficient  use  of 
the  severity  reduction  simulation  can  be  observed.  The  knapsack  heuristic  generates  a 
set  of  local  portfolios  out  of  modified  base  portfolios,  and  seeking  to  identify  improvement 
in  those  base  portfolios.  This  iterative  replacement  of  variables  with  those  having  a  high 
benefit-cost  ratio  enables  new  interactions  to  appear  while  feasibility  is  maintained.  This 
effective  method  of  identifying  local  improvement  may  help  identify  portfolios  that  contain 
synergistic  interactions. 
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Test  Portfolios 
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Figure  3.7  Example  of  Knapsack  Heuristic  Generation  of  Test  Portfolios 

3.3.2. 1  A  Knapsack  Heuristic  Example.  This  example  uses  actual  controls, 
their  individual  severity  reductions,  and  estimated  cost.  As  displayed  in  Figure  3.7,  the 
number  of  selected  items  in  the  initial  base  portfolio  indicates  the  number  of  test  portfolios 
generated  for  the  local  region.  A  new  test  portfolio  is  generated  by  setting  each  1  equal  to 
0  and  placing  the  remaining  budget  slack  into  other  unselected  controls  to  fill  the  budget 
keeping  the  portfolio  feasible.  There  are  four  possible  trends  for  the  unselected  controls  to 
assume  in  the  test  portfolios  generated  by  the  method.  First  (indicated  by  *1  in  the  table 
from  Figure  3.7)  a  control  may  be  selected  infrequently,  such  as  the  case  when  an  expensive 
control  is  unselected.  A  second  possible  trend  is  seen  in  note  *2  (in  the  table  from  Figure 

3.7) ;  this  control  is  never  selected  often  due  to  high  cost  and  low  reduction.  The  third, 
and  most  common  trend,  occurs  when  a  control  is  selected  because  of  a  high  benefit-cost 
ratio,  but  this  is  dependent  on  available  slack  budget  (see  note  *3  in  the  table  from  Figure 

3.7) .  The  final,  and  not  displayed,  trend  is  the  always  selected  control.  Unselected 
variables  in  the  lowest  cost  category  with  high  benefit-cost  can  be  selected  with  priority 
for  each  additional  test  portfolio.  Each  selected  variable,  as  indicated  by  a  1  in  the  test 
portfolio  column,  initiates  the  formation  of  test  portfolios.  The  values  returned  for  each 
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test  portfolio  may  show  improvement;  estimates  about  the  mean,  standard  deviation  and 
confidence  intervals  indicate  any  statistical  difference  in  the  portfolios. 

3.4  Methodology  Summary 

This  methodology  enables  the  Safety  Center  to  incorporate  ASIST  analysis  and  Risk 
Management  Information  System  (RMIS)  data  into  a  samples  that  demonstrate  accident 
sets  based  on  an  accepted  standard  of  severity.  The  Severity  of  Losses  model  defines 
the  factors  of  severity  that  are  most  important  for  a  balanced  evaluation  of  severity.  The 
Severity  Reduction  simulation  evaluates  each  portfolio  of  controls  and  is  able  to  provide  the 
90%  confidence  interval  for  the  expected  mean  Severity  Reduction.  Using  Direct  Search 
techniques  to  make  the  best  use  of  the  simulation  results,  the  objective  values  returned 
by  the  simulation  are  evaluated  using  a  multiple  choice  knapsack  heuristic.  The  following 
chapter  explores  the  results  of  this  methodology  applied  to  the  control-portfolio  selection 
problem. 
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IV.  Severity  Reduction  Analysis 


4.1  Top-Level  Portfolio  Selection  Overview 

This  research  develops  a  tool  that  enables  the  Army  Safety  Center  to  select  a  port¬ 
folio  of  controls  that  reduces  severity  of  aviation  accidents  by  50%;  as  an  example,  use 
the  force  modernized  aircraft,  AH-64,  CH/MH-47,  OH-58D,  and  UH/MH-60.  This  goal, 
set  by  the  Under  Secretary  for  Acquisition  and  Technology,  Army,  involves  incorporat¬ 
ing  the  risk  management  process  with  available  accident  investigation  information  and 
ASIST-identified  hazards  and  controls.  ASIST  first  performed  a  hazard  analysis  that 
both  identified  hazards  and  the  accident  cases  involving  them.  Next,  ASIST  looked  at 
each  accident  case  and  the  hazards  contributing  to  it  and  assigned  what  percent  of  the 
total  cost,  casualties,  and  frequency  of  the  accident  each  hazard  was  responsible  for.  These 
percent  contribution  values  quantify  the  preventative  components  of  each  accident.  The 
controls  generated  by  ASIST  are  tailored  to  fit  individual  hazards  and  eliminate  their  influ¬ 
ence  on  the  fleet.  ASIST  hypothesized  effectiveness  values  for  every  control-hazard  pair; 
these  unique  pair  combinations  allow  controls  to  impact  multiple  hazards  with  different 
values.  Meeting  the  Army  goal  requires  examining  selected  sets  of  controls,  portfolios  of 
controls ,  and  finding  the  combination  that  returns  the  greatest  reduction  in  accidents  for 
a  given  budget  amount. 

Selecting  an  individual  control  or  obtaining  a  ranked  list  of  top  controls  requires 
an  understanding  of  what  makes  a  control  effective.  Speaking  outside  of  the  Severity 
Reduction  Simulation  for  a  moment,  if  a  specific  type  of  accident  occurred  frequently,  for 
example  a  tree  strike,  the  following  actual  hazard  statement  would  capture  the  occurrence 
of  such  an  event: 


Table  4.1  Example  Hazard  Statement 


Hazard  H60-06 


Loss  of  situational  awareness  (as  a  result  of 
distance  estimation,  varying  workload,  environ¬ 
mental,  and  visual  issues)  while  maneuvering  in 
close  proximity  to  trees  or  objects  may  result  in 
the  aircraft  striking  the  trees  or  objects. 
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Several  controls  proposed  by  ASIST  aid  the  elimination  of  this  hazard.  The  Safety 
Center  generally  assumes  that  a  hazard  will  not  be  entirely  eliminated;  however,  it  is 
possible  to  nearly  eliminate  some  types  of  hazards  by  implementing  tractable  controls. 
Specific  controls  that  impact  hazard  H60-06  are  listed  in  Table  4.2. 


Table  4.2  Controls  that  Impact  Hazard  H60-06 


Control 

Control  Statement 

Effectiveness 

H60-C03 

Change  ATM  to  establish  new  flying  hour  cat¬ 
egory  for  individual  task  flight  training  hours 
and  resource  individual  training. 

0.25 

H60-C06 

Establish  and  sustain  crewchief’s  “school 
house”  training  program 

0.35 

H60-C07 

Establish  crew  coordination  sustainment  pro¬ 
gram 

0.5 

H60-C08 

Establish  standards  for  and  resource  a  4th 
crew  member  for  high  workload  ops 

0.45 

H60-C10 

Implement  a  change  to  the  flight  control  sys¬ 
tem  to  improve  aircraft  stability  and  control 
in  low  speed  flight  (Attitude  Command  Atti¬ 
tude  Hold) 

0.5 

H60-C11 

Increase  the  available  aircrew  experience 

0.25 

H60-C16 

Develop  and  field  a  proximity  warning  system 
(Virtual  Rotor  Disk) 

0.7 

H60-C24 

Develop  and  install  new  Night  Vision  Systems 

0.5 

This  table  indicates  the  numerous  possible  'solutions’  to  the  hazard  problem  stated 
above.  Each  hazard  within  the  system  has  at  least  one  control  that  may  be  selected 
to  reduce  its  likelihood  or  severity.  Generally,  the  more  controls  selected  to  reduce  a 
single  hazard,  the  less  frequently  it  will  occur.  The  conscious  selection  of  controls  involves 
analyzing  accident  cases  to  identify  the  most  frequent  and  most  severe  hazards,  observing 
the  controls  that  act  to  reduce  these  particular  hazards,  and  simulating  the  interactions  of 
simultaneously  applied  controls. 

This  chapter  addresses  the  percent  severity  reduction  output  obtained  by  simulated 
portfolios  and  identifies  valuable  insights  from  the  reductions  gained  by  these.  The  search 
procedure  first  explores  the  output  obtained  from  an  integer  programming  solution,  then  it 
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investigates  the  local  region.  The  goal  was  to  obtain  a  single  optimal  portfolio  of  controls; 
however,  this  analysis  displays  the  actual  plethora  of  reasonable  control  portfolios  with  an 
explanation  of  the  response  and  the  justification  for  future  recommendations. 

4-2  The  Integer  Problem 

This  problem  takes  input  from  the  Severity  Reduction  Simulation  and  focuses  on 
returning  a  good  starting  solution.  The  analysis  begins  with  the  single  control  portfolios, 
then  the  formulation  of  the  problem,  and  finally  insights  gained  by  this  process. 

4.2.1  Individual  Control  Effectiveness.  Implementing  single  control  portfolios 
allowed  each  control  to  be  evaluated  to  determine  its  mean  percent  severity  reduction  for 
blocks  of  100,000  flight  hours  over  five  years  in  the  force  modernized  aircraft  addressed. 
Controls  are  designed  to  reduce  accidents  for  a  particular  airframe;  hazards  may  affect 
multiple  airframes.  The  percent  composition,  by  airframe,  of  the  ASIST-evaluated  acci¬ 
dent  cases,  FY94  through  FY98,  reveals  the  possible  accident  draw  likelihood  as  depicted 
in  Figure  4.1.  Estimation  of  the  severity  contribution  of  each  airframe  requires  a  common 
factor,  such  as  percent  of  total  hours  flown  by  a  specific  airframe,  to  combine  severity 
scores.  It  may  be  estimated,  that  the  percent  of  actual  incidents  in  each  airframe  is  a 
better  standard  by  which  to  aggregate  severity  than  the  percent  total  hours  flown  in  a  year. 
From  the  information  contained  in  this  chart,  the  cumulative  distribution  representing  the 
percentiles  explains  the  selection  of  possible  accidents. 

An  individual  control’s  severity  reduction  must  be  less  than  the  affected  aircraft’s 
percent  composition  of  the  data  set  because  if  it  was  possible  to  decrease  all  accidents  for 
that  type,  the  overall  severity  reduction  is  expected  to  be  same  as  percent  composition.  A 
specific  hazard  may  cause  several  incidents  within  a  selected  set  of  data,  the  simple  relative 
volume  of  accidents  caused  by  a  particular  hazard  can  indicate  its  severity.  By  selecting 
a  control  that  reduces  these  most  commonly  occurring  hazards,  the  overall  severity  is 
decreased.  When  an  individual  control  is  applied,  it  reduces  the  accident  severity  for  that 
type  of  airframe.  The  best  reduction  found  by  this  study  for  any  single  control  portfolio 
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Composition  of  Accident  Data  Set 


H-47 

23% 


Figure  4.1  Cumulative  Composition  of  the  Data  Set  by  Airframe 

is  less  than  4%  over  all  the  force  modernized  aircraft.  The  single  control  reductions  serve 
to  estimate  the  possible  multiple  control  portfolios. 

4.2.2  Problem  Formulation.  This  problem  solves  the  surrogate  objective  function 
for  an  estimated  Severity  Reduction  value.  These  surrogate  function  coefficients  were 
derived  by  finding  the  mean  reduction  when  a  portfolio  consisted  of  a  single  control.  The 
budget  serves  as  the  primary  knapsack  resource  constraint.  The  additional  constraints  in 
this  integer  program  serve  to  eliminate  the  application  of  overlapping  controls.  Insights 
into  the  process  has  led  the  Safety  Center  to  remove  a  few  decision  variables  from  the  set, 
these  variables  will  always  equal  zero.  There  are  353  variables  and  15  constraints.  The 
Xi  variables  are  the  binary  control  selection  variables,  the  Ri  variables  are  the  reduction 
values,  and  the  Ci  variables  are  the  cost  of  individual  controls.  The  objective  is  to  estimate 
the  largest  percent  reduction  obtainable  from  the  combination  of  controls.  The  model  is: 
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Maximized  =  ^2Ri  ■  Xi 

(SLP) 

subject  to 

Ci-Xi 

<  Budget 

AHQ4C22  +  ,47/64(726 

<  1 

A7/64C52  +  Aff  64(753 

<  1 

7/47(787  +  7/47(7124 

<  1 

7/47(7105  4-  H47C127 

<  1 

7/47(752  +  7/47(7127 

<  1 

7/47(7103  +  7/47(7127 

<  1 

7/47(709  +  7/47(710 

<  1 

7/60(734  -  7/60(744 

<0 

7/60(725  +  7/60(746 

<  1 

7/60(716  +  7/60(720 

<  1 

7/60(720  +  7/60(721 

<  1 

7/60(715  +  7/60(724 

<  1 

7/60(717  +  7/60(744 

<  1 

3,47/64(765  -  ,47/64(766  -  ,47/64(767  -  ,47/64(769 

<  o 

Xi  =  {  0,1} 

Solving  this  surrogate  linear  program  returns  an  appropriate,  reasonable  starting 
portfolio  given  a  budget.  Discussion  with  the  Army  Safety  Center  led  to  the  establishment 
of  a  $1  billion  budget.  Sensitivity  analysis  about  this  fixed  budget  has  been  performed 
for  a  wide  range  of  cost  ($M)  because  the  controls  selected  to  be  funded  by  the  Army 
are  not  based  on  filling  a  given  budget,  but  meeting  the  set  reduction  goals.  Therefore, 
acceptance  of  a  budgeted  portfolio  of  controls  depends  on  the  quality  of  the  contents  of 
the  items  in  the  Program  Objective  Memorandum  (POM).  The  Safety  Center  budget 
allocations  for  the  application  of  controls  are  based  not  on  overall  cost,  but  rather  on  likely 
outcome  received. 

The  linear  program  assumes  an  additive  objective  function.  Using  Frontline  System’s 
Premium  Solver  Platform,  to  find  the  solution  to  the  linear  program  given  by  SLP,  the 
optimal  base  portfolio  is  found.  It  can  be  seen  through  testing  (see  Figure  4.2)  that  the  sum 
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of  single  control  portfolios  is  not  necessarily  equal  to  the  severity  reduction  value  obtained 
from  the  portfolio  containing  the  same  controls.  Therefore,  the  value  of  the  surrogate 
objective  function  is  an  ordinal  representation  of  the  magnitude  of  severity  reduction,  not 
an  actual  expected  percent  severity  reduction. 


Figure  4.2  Demonstration  of  Inadequacies  of  Surrogate  Objective  Value  Compared  to 
the  Simulated  Value 


From  Figure  4.2  the  base  portfolio  (□)  consists  of  159  controls  selected  to  maximize 
percent  severity  reduction.  This  portfolio  returns  an  expected  severity  reduction  of  35 
percent.  For  comparison,  several  potential  starting  portfolios  (A)  were  derived  using 
various  resource  allocation  techniques,  including  a  cost-effectiveness  ratio,  best  individual 
severity  reduction,  and  largest  number  of  controls  method.  Another  comparison  standard 
used  the  integer  program  with  surrogate  objective  function  to  generate  different  portfolios 
(♦)  by  varying  the  budget  constraint.  The  following  chart  identifies  the  predicted  surrogate 
reduction  in  severity  (x-axis)  versus  the  actual  observed  mean  reduction  in  severity  (y- 
axis).  These  clearly  sub-optimal  methods  of  completing  the  base  portfolio  indicate  that, 
although  the  surrogate  function  is  known  to  be  nebulous  in  its  definition,  it  is  an  ordinal 
representative  of  the  actual  percent  severity  reduction  values. 
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4.3  Optimal  Region  Search 

The  base  portfolio  resulted  from  fixing  the  budget  to  one  billion  dollars  in  the  resource 
constraint.  Given  this  initial  solution,  this  study  examines  the  improvement  gained  by 
interchanging  particular  controls  and  searching  in  the  local  regions.  The  method  for 
searching  the  region  near  the  base  portfolio  removes  each  selected  control  individually  and 
utilizes  the  allocated  budget  to  purchase  the  unselected  control(s)  that  have  the  highest 
cost-effectiveness. 

The  knapsack  heuristic  modified  as  described  in  Chapter  Three  utilizes  this  base 
portfolio  to  generate  159  new  test  portfolios  to  examine  for  improvement.  The  results 
of  this  test  indicate  extremely  small  variation  in  the  value  range;  no  statistically  different 
portfolios  resulted  from  this  first  examination.  Figure  4.3  indicates  the  surrogate  objective 
value  plotted  versus  the  actual  percent  severity  reduction  observed  by  the  simulation.  This 
portfolio  solution  displays  a  high  degree  of  degeneracy  arising  from  minute  differences  in 
individual  control  reductions.  The  maximum  surrogate  value  frontier  appears  at  0.75 
because  the  surrogate  optimal  function  was  found  and  multiple  test  portfolios  with  minimal 
surrogate  value  differences  exist  near  this  value. 

Sensitivity  analysis  to  budget  levels  was  then  examined.  In  an  investigation  of 
portfolios  at  $750  million  and  $1.5  billion,  very  similar  output  occurred  about  the  local 
regions,  see  Figure  4.4.  Although  the  surrogate  objective  function  increased,  the  actual 
severity  value  observed  by  the  simulation  indicated  little  increase  for  the  $750  million 
budget  increase.  These  regions  also  displayed  a  highly  degenerate  base  portfolio. 

The  level  of  degeneracy  led  to  a  closer  inspection  of  control  cost  and  how  its  increase 
affects  the  portfolio  selection.  For  example,  if  all  of  the  controls  costing  over  $19  million 
are  eliminated  from  consideration,  the  resulting  portfolio  generated  from  this  constrained 
problem  reveals  selection  preference.  The  order  of  magnitude  differences  between  cost 
make  selection  of  high  budget  controls,  those  over  $20  million,  very  rare. 

Examination  of  high-performing  controls  also  provide  insight  into  the  types  of  con¬ 
trols  returning  maximum  effectiveness.  The  cost-reduction  comparison  table  that  follows 
identifies  any  effectiveness  trends  related  to  cost.  These  mean  and  observed  reductions 
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Figure  4.3  Base  Portfolio  Exploration  of  Local  Region,  a  Complete  Perspective 


Figure  4.4  Budget  Sensitivity  about  Base  Portfolio:  Low  and  High  Cost  Portfolio  Re 


are  shown  with  two  significant  digits  to  demonstrate  the  lack  of  precision  offered  by  the 
simulation.  Observe  the  similarity  when  all  observations  of  each  type  are  averaged;  the 
only  dissimilarity  occurs  for  the  $0.05  million  category,  the  category  that  contains  the 
largest  number  of  observations  and  the  largest  number  of  0.0%  reductions.  When  seeking 
to  maximize  reduction,  the  lowest  reductions  are  not  likely  selected;  therefore,  observe 
the  best  single  percent  reduction  observations.  It  is  reasonable  to  expect  significant  dif¬ 
ferences  in  the  greatest  reduction  achieved  by  a  single  control  in  each  cost  category,  but 
there  is  less  than  1%  difference  between  four  of  the  five  categories.  Only  $0.5  million 
controls  show  increased  severity.  Nonetheless,  selecting  at  most  one  control  for  each  type 
of  aircraft  would  rarely  be  seen  in  a  portfolio,  so  the  five  best  controls  from  each  category 
have  been  evaluated  to  indicate  their  relative  severity  reduction.  The  last  two  columns 
of  the  cost-reduction  comparison  table  indicate  the  percent  range  of  severity  reduction  to 
expect  for  the  top  five  controls.  The  $200  million  category  and  the  $0.05  million  category 
particularly  draw  attention  by  indicating  that,  on  one  hand,  for  $1  billion  a  10%  reduction 
in  severity  is  possible  while,  on  the  other  hand,  a  10%  severity  reduction  is  also  possible 
costing  only  $0.2  million,  when  the  functions  behave  as  modeled. 


Table  4.3  Effectiveness  Trends. 


Control  Cost 

Number 
of  Controls 

Severity  Reduction 

All  Avg 

Best 

Top  5  Total 

Top  5  Avg 

$200,000,000 

49 

0.0044 

0.035 

0.10 

0.020 

$  90,000,000 

66 

0.0046 

0.039 

0.15 

0.030 

$  19,000,000 

87 

0.0032 

0.036 

0.13 

0.027 

$  500,000 

31 

0.0040 

0.014 

0.057 

0.012 

$  50,000 

120 

0.0019 

0.031 

0.099 

0.020 

4.3.1  Cost-Benefit  Insights.  The  analysis  examining  the  similar  reduction  in 
severity  for  controls  in  significantly  different  cost  ranges  reveals  the  most  significant  insight 
to  this  study.  Exhausting  a  specified  budget  should  not  be  the  objective.  The  Army  Safety 
Center  desires  the  capability  to  recommend  a  “One  to  Goal  list”  composed  of  the  number 
of  specific  controls  it  takes  to  obtain  a  50%  reduction  in  severity.  To  rank  the  Safety 
Center’s  “One  to  Goal”  list,  while  still  accounting  for  the  interactions  only  revealed  by  the 
simulation,  expensive  controls  must  be  selected  as  a  result  of  known  performance  versus 
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Figure  4.5  Plot  of  Individual  Control  Severity  Reductions  by  Control  Cost 

cost.  Rarely  would  a  single  $200  million  control  reduce  severity  greater  than  400  $0.5 
million  controls,  even  including  decreasing  marginal  returns.  The  orders  of  magnitude 
difference  in  cost  estimation  often  complicate  selecting  the  best  set  of  controls. 

The  actual  portfolios  observed  by  this  research  indicate  no  statistical  difference  be¬ 
cause  of  this  problem;  as  an  inexpensive  control  is  removed,  another  replaces  it.  The  final 
cost  of  these  controls  are  exactly  the  same,  and  even  though  the  severity  reduction  may 
be  similar,  the  overall  mean  cannot  change  significantly. 

4-3.2  Constrained  Searching.  The  previous  results  indicate  a  need  to  examine 
controls  when  a  tighter  budget  is  specified.  The  orders  of  magnitude  difference  between 
some  control’s  cost  eliminates  any  ability  to  capture  the  sensitivity  of  the  higher  cost  con¬ 
trols.  The  small  variations  in  effectiveness  between  low-cost  controls  hides  any  significant 
improvements  from  observation.  Addition  of  a  constraint  limiting  the  selection  controls  to 
those  with  a  certain  budget  enables  the  surrogate  integer  program  to  search  more  specifi¬ 
cally  to  identify  the  best  control  combinations.  The  two  regions  of  consideration  evaluated 
separately  in  the  problem  include  evaluating  only  controls  that  cost  less  than  $1  million 
and  evaluating  controls  that  cost  more  than  $1  million.  This  partition  was  selected  after 
observing  individual  control  impact  on  severity  reduction;  the  marginal  cost-effective  re¬ 
duction  in  severity  points  out  extreme  disparity  between  the  best  single  control  at  $0.05 
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million,  0.014  M/%,  and  the  top  single  $200  million  control,  64.5  M/%.  The  overall 
cost  of  all  353  controls  examined  totals  $17,414,500;  controls  less  than  $1  million  make  up 
43%  of  the  number  of  controls  (151)  but  less  than  1%  of  the  possible  total  cost  ($21. 5M). 
Generating  a  new  portfolio  based  on  these  diverse  selection  variables  involves  dividing  the 
decision  variables  into  two  sets,  the  high-cost  controls  (over  $1  million)  and  the  low-cost 
ones  (under  $1  million).  The  individual  cost  sensitivities  are  extremely  different  between 
the  problems,  therefore,  solve  the  low-cost  problem  first.  Aggregation  of  the  new  insights 
to  the  current  problem  provides  a  new  solution  with  less  repetitive  control  switching. 

The  large  set  of  possible  controls  allows  a  portfolio  generation  method  to  distribute 
funds  efficiently  to  within  $19M  of  the  budget  limit,  but  at  this  point  the  remainder  of 
the  budget  is  allocated  to  inexpensive,  and  sometimes  inefficient,  controls.  The  purpose  of 
this  subproblem  is  to  simplify  the  selection  variable  set,  eliminating  less  effective  controls 
thereby  forcing  the  remaining  budget  to  be  allocated  to  the  more  effective  controls.  The 
low-cost  control  integer  program  introduces  a  constraint  where  Cf  •  X\  =  0  for  all  i 
with  Ci  >  $1  million.  In  addition,  the  resource  constraint  for  the  budget  will  have  an 
adjusted  right  hand  side  with  an  appropriate  range  for  the  low-cost  controls,  $2  million 
to  $10  million.  This  maximum  range  is  derived  by  summing  the  cost  of  the  strongest 
controls,  those  with  an  individual  severity  reduction  greater  than  0.001.  The  method  for 
selecting  effective  controls  enabled  the  possible  control  set  to  be  reduced  by  77  inexpensive 
controls. 

Analysis  of  high-cost  controls  reveals  that  of  the  total  202  controls  that  cost  greater 
than  $1  million,  the  cost  and  effort  to  analyze  each  alternative  and  reduce  the  possible 
set  outweigh  the  benefit  of  the  results.  The  integer  program  effectively  selects  essential 
controls  to  accommodate  the  size  of  the  control  set.  Consequently,  no  additional  constraints 
for  high-cost  controls  have  been  added  to  the  problem. 

4.3.3  Expanding  the  search  to  a  second  base  portfolio.  Using  the  constraints 
generated  by  the  cost-partitioned  problem,  the  integer  program  recommends  a  secondary 
base  portfolio.  This  new  potential  control  set  forces  the  local  knapsack  search  algorithm 
to  select  higher  cost  controls  to  exchange  when  finding  test  portfolios.  Eliminating  extra- 
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neous,  less  effective,  controls  requires  different  controls  to  be  interchanged  and  the  local 
search  region  expanded.  The  secondary  base  portfolio  contains  103  controls,  which  is  56 
fewer  than  the  first  base  portfolio,  and  observes  a  mean  severity  reduction  within  1%  (less) 
of  the  first  base  portfolio.  The  cost  of  the  secondary  base  portfolio  is  not  more  than  $10 
million  less  than  the  original  base  portfolio  with  budget  constraint  at  $1  billion. 

Enabling  the  local  search  to  incorporate  a  wider  range  of  controls  reveals  additional 
search  regions  but,  most  importantly,  additional  insight  into  the  performance  of  high- 
cost  controls.  The  expected  percent  severity  reduction  for  this  region  spans  3.5%.  No 
portfolios  in  this  set  perform  better  than  the  best  portfolios  at  the  original  base  region. 
The  results  of  this  test  and  sensitivity  about  this  region  do  not  indicate  additional  tests 
and  exploration  are  needed. 

4-4  Selecting  the  Optimal  Portfolio 

The  preceding  search  procedures  guide  the  identification  of  the  recommended  port¬ 
folio  region  and  bracket  a  wide  range  of  “good”  portfolios.  The  actual  selection  of  a  single 
portfolio  provides  some  insight  into  the  process.  As  noted  in  the  previous  sections,  many 
portfolios  generated  similarly  to  those  in  this  set  could  represent  good,  possible  options 
for  the  decision  makers  at  the  Safety  Center  to  examine;  however,  the  real  insight  from 
these  portfolios  focuses  on  the  general  response  of  the  system.  Single  control  portfolios 
evaluated  by  this  simulation  tend  to  return  extremely  small  percent  reductions.  One  of 
the  best  observed  generated  only  a  3.5%  reduction,  that  being  control  H60-C34  (addition 
of  Digital  Source  Collector  (DSC),  envelope  cueing  (exceedences),  and  notice  to  pilot  of 
exceedences/crew  monitor) .  Single  control  portfolios  much  more  frequently  return  severity 
reductions  around  0.03%  such  as  AH64-C30,  acquire  crashworthy  ERFS  (Extended  Range 
Fuel  System). 

Examining  portfolios  across  the  range  of  budget  values  displays  an  increasing  Severity 
Reduction  with  increasing  budget,  see  Figure  4.6.  Unfortunately,  as  the  budget  increases, 
the  amount  of  reduction  per  dollar  spent  decreases.  Single  controls,  when  applied  together 
as  portfolios,  display  diminishing  marginal  returns.  This  finding  is  important,  but  not 
unexpected  based  on  the  discussion  in  Section  3.2.3,  because  none  of  these  simulated 
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observations  meet  the  goal  of  50%  reduction  in  severity.  Previously,  ASIST  has  been  able 
to  meet  the  50%  goal  by  evaluating  controls  individually  and  assuming  that  every  control 
acted  independently  of  others;  the  simulation  results  from  this  study  disagree  with  that 
assumption.  These  results  indicate  that  as  additional  controls  are  added,  severity  decreases 
at  a  decreasing  rate.  The  purely  additive  model  does  not  represent  this  phenomenon. 
These  diminishing  marginal  returns  suggest  that  spending  an  additional  $1  billion  may 
not  even  return  a  significant  decrease  in  severity.  More  investigation  into  this  aspect  of 
the  problem  is  needed. 


Simulated  Percent  Severity  Reduction  with 
increasing  Portfolio  Budget 


Portfolio  Budget  ($M) 


Figure  4.6  Comparison  of  Additive  Severity  Reduction  and  Actual  Simulated  Severity 
Reduction 

There  are  two  elements  to  selecting  an  acceptable  portfolio:  first,  good  controls  must 
be  selected  and  second,  interactions  must  be  used  to  achieve  maximum  reduction.  An 
acceptable  portfolio  meets  all  feasibility  requirements  defined  in  the  constraints  of  the 
integer  program. 

4,4.1  Finding  better  controls.  The  results  of  this  analysis  of  portfolios  reveal  that 
individual  controls  that  reduce  the  most  severity  effectively  reduce  the  most  frequently 
occurring  or  the  most  severe  hazards.  In  a  way,  the  controls  that  show  highest  reduction  are 
the  controls  that  best  fit  the  definition  of  a  control:  an  implemented  action  that  reduces  the 
likelihood  or  severity  of  a  hazard  occurring.  The  first  result  should  intuitively  make  sense, 
to  obtain  the  greatest  reduction,  fix  the  problems  that  occur  most  often.  Control  H60-C34 
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is  mentioned  above,  and  this  particular  high-cost,  material-driven  solution  returned  good 
reduction  in  severity  primarily  because  it  directly  reduced  AVN-01  hazards  which  appear 
27  times  in  the  accident  data  set.  A  possible  control  may  also  decrease  severity  by  a 
substantial  amount  if,  although  an  specific  hazard  occurs  infrequently,  the  results  of  it  ever 
occurring  are  catastrophic.  Note  hazard  AH64-10,  failure  of  the  yellow  engine  harness  can 
cause  the  HMU  to  command  the  engine  power  to  either  underspeed  or  overspeed  resulting  in 
degraded  engine  performance  and  possible  aircraft  damage.  This  hazard  only  appears  twice 
in  the  current  accident  data  set;  however,  implementation  of  control  AH64-C38,  mandatory 
replacement  with  new  yellow  harness,  entirely  eliminates  this  hazard.  Consequently  this 
single  control,  has  a  severity  reduction  of  1.5%,  one  of  the  best  25  individual  controls. 

If  selecting  an  individual  control,  note  the  specific  cost,  casualty,  and  frequency 
contributions  of  the  affected  hazard.  A  single  control  may  only  eliminate  the  severity  of 
cost  and  leave  casualties  at  the  current  level.  The  contribution  break  down  enables  a  user 
to  figure  out  what  severity  issues  are  reduced  by  a  specific  hazard.  The  affect  of  different 
hazards  contributing  to  the  same  incident  leads  to  the  incorporation  of  multiple  controls 
as  a  better  method  for  reducing  severity. 

4.4.2  Finding  better  portfolios.  When  evaluating  the  surrogate  objective  func¬ 
tion  to  initialize  the  search  region,  the  result  of  this  additive  model  indicated  that  although 
the  coefficients  accurately  represent  single  controls,  their  combined  output  differed  signifi¬ 
cantly.  Each  expected  severity  reduction  observed  as  output  of  the  model  under- achieves 
the  projected  percent  reduction.  The  combined  hazard  effectiveness  value  rationally  re¬ 
duces  the  effect  of  two  controls  acting  on  one  hazard;  this  is  one  cause  of  over  estimation. 
Due  to  innumerable  control  combinations,  simulation  output  must  be  used  in  conjunction 
with  search  methods  to  find  and  accurately  identify  effective  control  combinations. 

The  differences  between  the  mean  severity  reductions  that  span  $100  million  through 
$1.5  billion  is  approximately  0.05.  The  90%  confidence  interval  about  the  $95  million 
budget  is  (0.23,  0.38);  the  90%  confidence  interval  about  the  $1.5  billion  portfolio  is  (0.28, 
0.41).  These  differences  arouse  interest  about  the  controls  contained  in  the  low-cost 
budget.  An  analysis  of  the  difference  between  the  134  controls  in  the  low-cost  portfolio 
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and  the  159  controls  in  the  $1  billion  portfolio.  Note  the  small  difference  in  the  number 
of  controls,  closer  inspection  reveals  that  the  remaining  $900  million  funds  more  expensive 
controls.  When  these  individual  budgets  are  broken  down  by  DOTLMS  area,  percentage 
weight  of  the  budget  shifts  from  training  controls  in  the  low-cost  portfolio  to  materiel  in 
the  high-cost  budget. 


DTLOMS  Areas  for  $100M  budget 
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Figure  4.7  Budget  Proportions  for  DTLOMS  with  a  $100M  budget  portfolio 
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Figure  4.8  Budget  Proportions  for  DTLOMS  with  a  $1B  budget  portfolio 

The  cost  scale  differences  are  of  particular  significance,  showing  the  relative  differ¬ 
ences.  From  these  charts,  note  the  weighty  proportion  of  the  cost  spent  on  AH-64  materiel. 
The  UH-60  places  demands  on  the  budget  for  its  materiel  controls  when  funds  are  avail- 
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able,  but  under  a  lower  budget  uses  training  to  reduce  accidents.  These  bar  charts  provide 
insight  that  a  decision  maker  could  use  when  adapting  to  changing  budget  allocations. 

4.5  Results  Summary 

This  chapter  explored  individual  controls  and  the  responses  of  portfolios;  both  pro¬ 
vided  insights  for  decision  makers  to  consider.  None  of  the  observed  portfolios  attained 
the  goal  of  a  50%  reduction  in  severity;  however,  the  method  of  aggregating  all  data  into 
one  general  set  and  the  conservative  method  of  estimating  hazard  reduction  impacted  these 
results.  Based  upon  the  responses  attained,  the  Severity  Reduction  Simulation  suggests 
a  more  detailed  analysis  of  the  interactions  of  promising  portfolios  of  controls.  It  also 
suggests  focusing  portfolio  selection  on  a  careful  examination  of  low-cost  controls  within 
an  assumed  budget. 
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V.  Conclusions 


The  Army  Safety  Center  initiated  this  partnership  with  AFIT  to  better  analyze  and  an¬ 
swer  one  question:  how  can  we  meet  the  Secretary’s  goal  of  a  3a  reduction  in  accidents ? 
Since  this  time,  the  goal  has  become  more  specific,  by  defining  severity,  and  now  asks: 
what  are  the  best  controls  to  apply  to  obtain  a  50%  reduction  in  severity ?  This  research 
effort  integrates  several  operations  research  techniques,  incorporating  the  data  collected 
in  the  Risk  Management  Information  System  and  the  insights  of  experts  and  the  USASC 
leadership,  to  form  a  process  that  helps  the  Army  meet  their  goal.  Using  decision  anal¬ 
ysis,  Monte-Carlo  simulation,  linear  programming,  and  search  heuristics,  the  combined 
methodology  demonstrated  within  this  study  recommends  techniques  for  the  selection  of 
control  portfolios.  The  methodology  provided  by  this  research  assists  the  Army  Safety 
Center  in  accomplishing  their  mission;  it  takes  a  proactive  approach  to  hazard  analysis 
while  incorporating  risk  management  tools  with  the  ultimate  goal  of  “preventing  the  acci¬ 
dental  loss  of  America’s  most  precious  resources-  it’s  sons  and  daughters-  and  conserving 
[their]  materiel  resources”  (1). 

5.1  Overview  of  Results 

Incorporating  multiple  analysis  techniques  into  an  automated  system  provides  a  mea¬ 
sure  of  mean  severity  reduction  for  a  particular  portfolio  of  controls.  These  controls,  as 
identified  and  evaluated  by  ASIST,  decrease  the  likelihood  or  severity  of  an  accident. 
Insights  presented  in  Section  4.2.1  provide  ways  to  interpret  the  control’s  effectiveness. 
Although  there  are  many  ways  to  select  particular  portfolios  from  the  2353  =  1.8  x  10106 
possible  options,  the  integer  program  with  surrogate  coefficients  results  in  a  consistent 
measure  that  ordinally  represents  actual  portfolio  reductions.  Exploring  the  regions  sur¬ 
rounding  the  base  point  involves  generating  many  portfolios  in  the  local  region.  The 
problem  evolved  further  after  a  lack  of  sensitivity  to  cost  was  observed;  the  addition  of  a 
constraint  limiting  the  low-cost  controls  minimized  the  search  region  and  recommended  a 
slightly  different  alternative  base  portfolio.  No  single  optimal  portfolio  from  the  multi¬ 
tude  of  possible  controls  is  recommended,  however,  regions  for  reasonable  operations  can 
be  identified  using  this  methodology.  The  following  sections  identify  specific  criteria  for 
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selecting  portfolios  and  also  explain  further  steps  that  may  enable  this  methodology  to 
return  more  specific  values. 

5.2  Benefit  Added  by  the  Research 

This  research  represents  the  third  step  of  an  on-going  partnership  between  USASC 
and  AFIT;  portions  of  this  work  are  an  adaptation  and  enhancement  of  previous  work.  The 
points  presented  here  identify  the  unique  benefits  of  this  work.  Previous  work  addressed 
the  aviation  accident  severity  from  a  narrower  viewpoint;  this  study  takes  the  first  step 
toward  joining  multiple  systems  into  a  single  set  for  measuring  accident  severity.  The 
examination  of  portfolios  of  controls,  used  in  the  search  algorithm,  clearly  points  out  that 
the  system  displays  diminishing  marginal  returns  in  Severity  Reduction  with  respect  to 
cost.  In  the  course  of  the  effort,  errors  within  H-47  ASIST  database  were  detected,  and 
once  these  errors  are  rectified,  future  efforts  will  be  able  to  use  ASIST  CH/MH-47  data 
as  actual ,  rather  than  just  notional ,  data.  Increasing  the  usability  of  the  system  was  a 
primary  focus  early  in  the  effort.  At  the  conclusion  of  this  effort,  there  exists  an  automated 
severity  reduction  simulation  ready  to  be  integrated  into  ©RISK,  or  another  Excel-based 
simulation  software. 

Many  possible  integration  standards  were  considered  for  combining  multiple  systems 
into  a  more  Army-focused  model.  To  aggregate  accident  severity  by  system,  the  primary 
question  is:  “what  is  the  best  predictor  of  future  accidents,  by  type?”  Hours  flown  by 
aircraft  type,  hours  flown  per  month  per  aircraft,  straight  percent  composition  of  the 
fleet,  and  actual  past  accidents  are  all  measures  for  aggregating  accident  data.  The  Safety 
Center  desires  the  focus  to  remain  at  the  highest  level  possible,  the  Army-focused  approach, 
because  the  target  is  an  Army-level  goal  of  50%  reduction  in  severity.  To  best  answer 
this  question,  the  ASIST-evaluated  accident  cases  were  combined  into  a  single  set.  A 
Poisson-  based  random  variate  generated  possible  accident  numbers,  and  actual  cases  were 
randomly  selected  as  possible  accidents.  Therefore,  the  distribution  of  systems  selected  are 
based  on  the  percentage  of  each  aircraft  type  as  represented  in  the  database.  Aggregating 
by  this  factor  implies  that  future  accidents  will  tend  to  be  distributed,  over  aircraft  types, 
in  the  future  as  the  distributions  have  been  in  the  past  (ie.,  time-invariant). 
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The  original  ASIST  implementation  of  the  data  found  each  aviation  control  value, 
and  ranked  the  controls,  and  then  considered  cost  to  select  the  “One  to  Goal  List.”  This 
method  makes  several  assumptions,  including  the  additivity  of  severity  reduction.  This 
research  demonstrates  that  when  combining  controls  into  a  set,  if  their  total  reduction  is 
less  than  the  sum  of  the  individual  control  reductions,  a  shortfall  occurs.  Because  the 
Army  desires  to  achieve  50%  reduction  in  severity,  particular  attention  must  be  afforded 
to  the  assumption  of  additivity  of  severity.  As  noted  in  Section  4.3.1,  a  comparison  of 
cost  versus  effectiveness  consistently  displays  diminishing  marginal  returns.  Currently, 
simulation  techniques  most  effectively  incorporate  the  uncertainty  of  accident  occurrences 
with  the  possible  types  to  provide  expected  ranges  for  the  mean  severity  reduction.  These 
techniques  should  be  updated  as  necessary  to  keep  the  input  current  and  provide  the  best 
model  of  commander  preference  and  observation. 

Using  notional  data  for  the  ASIST-evaluated  information  on  the  H-47  airframe,  this 
study  to  modeled  the  complete  force-modernized  aircraft  branch  of  the  Army  severity 
model.  Corrected  data  for  the  H-47  database  is  forthcoming;  integrating  this  corrected 
information  involves  nothing  more  than  basic  spreadsheet  manipulation. 

Prior  to  this  effort,  a  good  simulation  methodology  was  in  place;  however,  this  highly- 
interactive  system  required  significant  man-hours  and  provided  plenty  of  opportunity  for 
human  error.  To  evaluate  portfolios,  this  research  developed  a  system  that  takes  a  dis¬ 
tribution,  a  0-1  portfolio  specifying  a  set  of  controls,  and  its  location  and  returns  a  mean 
severity  reduction.  This  system  is  described  in  more  detail  in  Appendix  B.  This  Sin- 
gle_Run  routine  was  developed  to  return  output  that  is  then  used  in  an  ©RISK  model. 
Although  each  simulation  takes  approximately  three  minutes  to  complete,  the  operator 
now  benefits  from  significantly  shortened  set-up  processing. 

The  Safety  Center  benefits  from  sponsoring  this  effort  by  obtaining  a  clarified  method¬ 
ology,  an  improved  database,  and  a  more  user-friendly  system.  The  Safety  Center  intends 
to  continue  this  partnership  and  further  expand  this  model.  There  is  also  the  possibility 
that  the  Value-Focused  Thinking  approach  may  be  used  across  services. 
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5. 3  Insights 

Obtaining  a  high  reduction  in  severity,  specifically  a  reduction  greater  than  40%, 
is  extremely  rare  using  this  model  with  the  current  data  set.  Because  the  Army  seeks 
to  reduce  severity  by  at  least  50%,  this  is  a  significant  problem.  A  hypothesized  reason 
for  the  sub-standard  reduction  in  severity  comes  from  the  actual  Severity  of  Losses  values 
returned  by  each  individual  repetition  within  an  iteration.  The  individual  repetition 
calculates  a  Severity  of  Losses  for  the  randomly  selected  accident  data,  then  it  applies  the 
selected  controls  and  obtains  the  new  Severity  of  Losses.  The  percent  difference  in  these 
values  is  the  actual  percent  severity  of  losses.  Although  this  method  of  calculation  returns 
the  specific  desired  metric,  the  actual  measured  Severity  of  Losses  values  are  lost  in  the 
calculation.  A  few  selected  runs  reveal  the  actual  Severity  of  Losses  for  the  extreme  cost 
portfolios:  a  high-cost  portfolio  provided  a  score  of  0.08  severity ,  while  a  low-cost  portfolio 
averaged  0.13.  These  portfolios  expected  a  mean  severity  reduction  between  0.12  and  0.24, 
for  the  low-cost  portfolio  and  between  0.37  and  0.49  for  the  high-cost  portfolio.  A  single 
block  of  100,000  flight  hours  is  generated  26  times  per  iteration,  for  the  approximately 
2,600,000  flight  hours  in  5  years  for  the  force-modernized  aircraft  in  the  study.  The 
simulation  from  these  100,000  horn  blocks  provides  the  90%  confidence  intervals  covering 
the  expected  percent  severity  reduction. 

The  current  framework  gives  commanders  a  multitude  of  possible  portfolios  from 
which  to  select.  Although  the  Safety  Center  originally  identified  controls  while  ignoring 
the  cost,  at  some  point  in  the  selection  process  cost  must  become  a  factor.  When  observing 
actual  full  portfolios,  there  are  many  factors  to  consider  such  as  the  airframes  affected,  the 
DTLOMS  areas  addressed,  and  the  cost  allocation.  The  tabular  data  presented  in  Figures 
5.1  and  5.2  show  a  single  portfolio  broken  down  into  significant  parts. 

This  table  displays  many  of  the  issues  that  a  commander  must  take  into  consideration 
when  selecting  a  portfolio  from  a  set  of  similar-return  options.  Due  to  the  overlapping 
confidence  intervals  about  the  mean  severity  reduction  for  portfolios  from  $100  million  to 
$1.5  billion,  the  Army  must  choose  how  to  allocate  their  funds.  If  portfolios  differing  by  $1 
billion  likely  return  comparable  reductions,  the  Army  could  decide  to  request  the  smaller 
budget  control  portfolio  and  have  a  better  likelihood  of  receiving  funding. 
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Figure  5.1  An  Actual  Portfolio  of  Controls,  Categorized  by  Airframe  and  DTLOMS  Area 
(part  1) 
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5.4  Future  work 

The  model  framework  presented  in  this  analysis  is  supported  by  the  necessary  theory; 
however,  the  desirable  recommended  numerical  results  do  not  appear.  In  any  input-output 
process,  careful  examination  to  ensure  the  process  uses  good  input  yields  the  promise  of 
improved  output.  These  recommendations  for  future  work  focus  on  improving  the  input 
data. 

Better  estimates  of  severity  would  go  a  long  way  to  improve  results.  To  better 
illustrate  what  severity  looks  like  for  each  airframe,  specialized  severity  functions  will 
increase  the  precision  for  severity  in  100,000  flight  hours.  A  possibility  for  this  is  to 
consider  severity  functions  that  incorporate  a  severity  value  of  1.0  within  the  range  of  values 
(see  Figure  5.3).  As  with  any  decision  analysis  study  that  incorporates  Value-Focused 


Figure  5.3  Example  Evaluation  Measure 


Thinking,  the  commander’s  preferences  need  to  be  updated  to  include  the  preferences  of 
the  most  senior  level  decision  maker.  Particularly  in  the  case  of  these  ever-changing 
accident-based  evaluation  measures,  a  point  must  be  made  to  reflect  the  Army’s  current 
goals  and  values. 

Controls,  although  designed  to  address  specific  hazards,  may  in  fact  impact  other, 
similar  hazards.  The  hazards  and  associated  controls  currently  in  use  have  all  been  de¬ 
veloped  by  airframe.  This  entire  research  effort,  as  well  as  on-going  Army  aviation  safety 
decisions,  are  based  entirely  on  this  set  of  data.  The  suspected  synergistic  interplay  be¬ 
tween  controls  and  hazards,  along  with  any  resultant  severity  reduction  (as  first  pointed 
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out  in  Section  3.3.1),  may  provide  additional  insights  as  to  where  to  most  effectively  spend 
the  safety  budget,  leading  to  better  safety  decisions.  It  is  worth  noting  that  intial  obser¬ 
vations  from  this  research  indicate  that  overlap  in  hazard  occurrence,  and  hence  overlap 
in  controls  designed  to  mitigate  those  hazards,  lead  to  an  overall  smaller  than  additive 
severity  reduction. 

By  defining  a  better  hazard  and  control  taxonomy,  elimination  of  redundant  hazards 
and  combining  overlapping  controls  would  serve  to  make  the  search  region  more  compact. 
This  feature  enables  significant  hazards  and  controls  to  be  highlighted.  Currently  a  pilot 
study  exists  that  addresses  the  hazards  associated  with  tree-strikes.  Apply  the  analyst’s 
current  test  case  as  an  example  of  how  although  ASIST  evaluated  them  well,  combining 
them  as  much  as  possible  to  create  more  robust  controls  and  hazards  will  make  the  program 
in  general  better.  Better  input  yields  better  output. 

As  with  any  analysis,  the  more  precise  the  input,  the  less  the  uncertainty  of  the 
output.  For  this  study,  ASIST  identified  controls  without  regard  for  the  possible  cost  of 
implementing  these.  Decision  makers  will  definitely  consider  cost  as  a  factor;  the  better  the 
cost  estimates,  the  more  realistic  these  portfolios  of  controls  can  be.  Because  the  current 
estimates  are  only  measures  of  magnitude,  it  is  important  to  refine  the  cost  estimates  to 
better  budget  controls. 

There  is  potentially  a  need  for  additional  research  to  normalize  the  systems  based  on 
something  other  than  flight  hours  because  the  likelihood  of  accidents  is  not  based  on  use, 
but  also  on  type  of  use. 

5.4-1  Expanded  Applications.  This  study,  although  it  specifically  addresses  Army 
aviation,  has  many  applications  for  other  services  as  well  as  the  aviation  industry.  The 
safety  community  commonly  categorizes  conditions  and  correction  factors  into  hazards  and 
controls.  By  incorporating  the  severity  measure,  identifying  hazards  and  controls,  and 
evaluating  severity  reduction,  this  methodology  expands  a  framework  that  may  be  used 
on  multiple  other  systems.  The  effort  to  modify  thinking  to  address  severity ,  rather  than 
accident  rate,  will  help  the  related  safety  organizations  to  identify  their  primary  concerns 
under  multiple  competing  objectives  for  quantifying  severity  of  accidents.  Support  for 
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severity  as  a  measure  has  been  developed  in  other  sectors  of  safety  research  to  include  (28). 
This  view  of  accident  safety  enables  leaders  and  commanders  to  make  better  decisions  by 
evaluating  the  effectiveness  of  controls  in  reducing  accident  severity. 

Adding  the  evaluation  of  all  accident  cases  in  terms  of  hazard  conditions,  as  the 
Army  is  now  proposing,  prepares  the  service  to  select  controls  by  this  quantitative  and 
qualitative  method.  The  Federal  Aviation  Administration  has  pursued  accident  safety 
with  a  hazard  focus  in  their  1998  study  “Safer  Skies:  A  Focused  Agenda”  which  identified 
the  important  factors  in  accidents  (12).  Their  approach  successfully  identified  focal  issues 
for  commercial  aviation  safety;  studing  possible  controls  and  a  quantitative  estimation  of 
effectiveness  is  the  next  logical  step.  This  expanded  case  study  for  the  Army  Safety  Center 
shows  how  hazard  and  control  data,  a  severity  value  model,  and  a  simulation  can  be  used 
with  heuristic  techniques  to  select  sets  of  controls  to  apply  for  the  reduction  in  aviation 
accident  severity. 

As  the  U.S.  Air  Force  also  seeks  to  improve  its  safety,  this  methodology  takes  the  focus 
off  of  possible  alternatives  and  places  it  on  reducing  the  important  factors  affecting  aviation 
accident  severity.  The  Army  has  integrated  multiple  rotary  wing  systems  to  obtain  a  single 
overall  severity  reduction  measure;  likewise,  the  Air  Force  can  use  safety  publications  to 
establish  a  multiple  objective  accident  severity  model  and  evaluate  its  systems.  The 
integration  of  bootstrapping  using  simulation  and  portfolio  selection  based  on  an  initial 
surrogate  objective  function  with  associated  extrapolated  regions  allows  commanders  to 
look  at  the  alternatives  as  evaluated  based  on  their  achievement  of  the  goal:  severity 
reduction. 

5. 5  The  Finale 

The  methodology  provides  a  defensible  way  that  individual  controls  may  be  priori¬ 
tized.  However,  overall  accident  interactions  cannot  be  simply  quantified  using  an  additive 
model,  a  simulation  technique  best  incorporates  the  uncertainty.  The  audacious  goal  that 
the  Army  set  may  be  achievable  for  individual  aircraft;  however,  this  study  reveals  that 
an  exorbitant  amount  of  funds  would  likely  not  consistently  produce  the  desired  results 
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overall.  Finally,  improvements  made  to  individual  parts  of  the  model  to  better  estimate 
inputs  can  only  result  in  improved  recommendations  and  possible  portfolios. 
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Appendix  A.  Severity  Functions 

The  support  for  the  severity  evaluation  measures  are  developed  in  Chapter  III;  this  ap¬ 
pendix  develops  the  single  dimensional  severity  functions  for  each  evaluation  measure.  A 
single  dimensional  value  function  is  constructed  such  that  the  preferred  level  receives  a  1, 
and  the  least  preferred  level  receives  a  0  (19:61).  Working  with  the  Army  Safety  Center, 
severity  is  modeled  similarly;  however,  the  least  severe  situation  is  most  desirable,  receiv¬ 
ing  a  zero.  Likewise,  the  most  severe  accident  situation  is  least  desirable,  receiving  a  one. 
This  description  is  clearly  understood  and  supported  by  the  leadership  at  the  Safety  Cen¬ 
ter.  The  following  description  of  individual  value  function  ranges  are  based  on  simulation, 
the  shape  of  the  curves  are  based  on  interviews  with  experts,  and  the  final  overall  approval 
given  by  the  ORSA  Division  Chief,  MAJ  Sperling. 

At  the  time  of  elicitation,  the  following  officers  at  the  Safety  Center  provided  input 
in  the  area  of  their  expertise  for  the  Severity  functions.  Casualties  and  total  costs  severity 
functions  were  developed  with  COL  Warren,  USASC  Deputy  Commander.  Unit  readiness 
severity  functions  were  developed  with  prior  battalion  commander,  LTC  Semmens,  USASC 
Executive  Officer.  Finally,  LTC  Gleisberg,  USASC  Judge  Advocate  General  developed  the 
environmental  damage  severity  functions  because  of  his  background  in  environmental  law. 
In  order  to  obtain  a  worst  case  range  estimate,  ten  thousand  simulated  blocks  of  100,000 
flight  hours  were  observed  to  find  the  maximum  range  for  each  measure. 

A.l  Casualties 

The  following  severity  functions  use  Army  officers’  preferences  and  Army  accident 
data  to  assess  the  severity  of  casualties  for  100,000  flight  hours  in  the  selected  Force  Mod¬ 
ernized  Aircraft. 

A.  1.1  Loss  of  Life.  The  loss  any  soldier  is  tragic.  Adequately  quantifying  the 
severity  of  a  lost  life  is  extremely  difficult.  COL  Warren  expressed  that,  to  the  Army, 
each  soldier  is  equally  important.  A  linear  relationship  between  lives  lost  and  severity 
best  describes  the  Army  perspective  (13:30). 
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From  observation  of  the  fatalities  present  in  the  data  addressed  by  ASIST,  slightly 
less  than  2  lives  lost  per  100,000  flight  hours  is  expected.  Simulation  shows  an  expected 
maximum  observable  number  of  fatalities  for  100,000  flight  hours  to  be  23.  Consequently 
the  range  of  incidents  for  fatalities  in  100,000  flight  hours  is  0  to  23  fatalities  in  100,000 
flight  horns  for  the  Force  Modernized  Aircraft. 
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Figure  A.l  Severity  Function  of  Evaluation  Measure  -  Fatalities 


A.  1.2  Permanent  Total  Disabilities.  Permanent  total  disabilities  occur  very 
infrequently.  In  fact,  there  are  no  observed  cases  of  permanent  total  disabilities  for  the 
AH-64,  the  H-47,  the  OH-58D  and  the  UH-60  during  FY94  through  FY98.  Occurrences 
of  this  type  of  casualty  did  occur  in  other  airframes  during  this  time  period,  and  they 
have  occurred  in  incidents  involving  the  force  modernized  aircraft  during  different  periods 
of  time.  For  the  value  model,  the  necessary  evaluation  of  the  severity  function  is  based 
on  preferences  and  experience  of  commanders.  Noting  the  severity  of  a  single  permanent 
total  disability  at  0.75,  an  increase  of  each  additional  total  disability  is  linear.  The  range 
for  permanent  total  disabilities  is  0  to  6  permanent  total  disabilities  based  on  historical 
data  in  other  time  periods. 

A.  1.3  Permanent  Partial  Disabilities.  This  severity  function  captures  the  impact 
of  permanent  partial  disabilities.  Similar  to  the  tragic  incidence  of  loss  of  life,  an  occurrence 


A-2 


of  permanent  partial  disabilities  effectively  causes  the  Army  to  lose  a  soldier.  The  inability 
to  serve  in  the  same  capacity  is  felt  by  the  Army,  and  each  permanent  partial  disability 
that  occurs  is  equally  as  severe  in  the  judgement  of  the  Army  (13:31).  The  ASIST  data 
does  contain  occurrences  of  this  type  of  casualty  and  its  severity  is  based  on  a  range  of  0 
to  4  permanent  partial  disabilities. 


+-> 
•  * 
j-i 
0> 
> 
U 
GO 


0  05  1  IS  2  25  3  35  4 

Permanent  Partial  Disabilities 


Figure  A. 3  Severity  Function  of  Evaluation  Measure  -  Permanent  Partial  Disabilities 


A.  1.4  Time  Incapacitated .  To  measure  the  loss  the  Army  experiences  from  a 

single  injured  soldier,  time  incapacitated  is  scored  in  terms  of  the  total  days  soldiers  spend 
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in  the  hospital  as  a  result  of  accidents.  The  Army  views  each  day  hospitalized  as  equally 
severe;  a  linear  response  describes  the  relationship  between  days  hospitalized  and  severity 
(13:31).  The  accident  data  and  simulation  suggest  the  range  for  time  incapacitated  to  be 
0  to  153  days  of  hospitalization  for  100,000  flight  hours. 


Days  Hospitalized 


Figure  A.4  Severity  Function  of  Evaluation  Measure  -  Time  Incapacitated 


A. 2  Unit  Readiness 

The  severity  functions  that  evaluate  unit  readiness  focus  on  the  battalion  level.  Eval¬ 
uating  the  impact  of  accidents  on  a  single  unit’s  readiness  means  developing  the  impact 
that  accidents  have  on  training  execution,  unit  morale,  and  equipment  availability.  Be¬ 
cause  the  model  evaluates  severity  for  100,000  flight  hours  for  the  force  modernized  aircraft, 
no  distinction  is  made  between  the  types  of  units  the  aircraft  belong  to.  This  enables  us 
to  focus  on  Army  impact  while  addressing  unit  issues.  The  information  that  these  sever¬ 
ity  functions  are  based  upon  was  elicited  from  LTC  Semmens,  a  prior  aviation  battalion 
commander.  This  input  provided  excellent  guidance  for  relative  relationships  to  severity 
concerning  the  issues  impacting  a  unit.  Consequently,  his  input  is  used  as  a  guide,  no 
specific  data  points  were  elicited;  future  expanded  studies  should  ideally  include  a  complete 
elicitation  from  a  group  of  battalion  commanders  (13:32). 
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A. 2.1  Training  Execution.  As  accidents  within  a  unit  increase,  a  commander 
may  reduce  the  training  frequency,  complexity,  and  realism  in  an  effort  to  reduce  additional 
accidents.  This  measure  combines  the  number  of  accidents  that  happen  within  a  unit  and 
the  class  of  those  accidents.  A  battalion  commander  may  tend  to  be  more  or  less  risk 
averse  depending  on  the  class  of  the  accidents.  It  is  estimated  that  he  would  be  as  risk 
averse  for  three  class  C  accidents  as  he  would  be  for  one  class  A  accident.  The  following 
provide  a  scale  for  measuring  one  accident  of  the  specified  class  (13:33): 

•  Class  A  accident  =  3 

•  Class  B  accident  =  2 

•  Class  C  accident  =  1 

The  category  for  evaluation  is  determined  by  accounting  for  the  class  of  the  accident 
multiplied  by  the  number  of  occurrences  within  the  unit.  For  example,  if  a  certain  unit: 
had  two  accidents  in  100,000  flight  hours,  one  class  A  accident  and  one  class  B  accident, 
the  training  execution  category  would  be  5.  If  another  unit  had  two  class  C  accidents,  the 
training  execution  category  would  be  2.  The  severity  assigned  to  each  category  can  be  seen 
in  figure  A.5.  The  largest  increase  occurs  between  categories  2  and  4.  This  indicates  that 
a  commander  would  be  slightly  risk  averse  if  a  single  class  C  or  class  B  accident  occurred  in 
their  unit;  however,  if  a  class  A  accident  or  multiple  class  B  and  class  C  accidents  occurred, 
the  commander  would  likely  become  more  risk  averse.  The  result  of  his  risk  aversion  is 
a  reduction  in  training  execution,  consequently  more  accidents  and  the  higher  the  class  of 
those  accidents  within  a  single  unit  the  greater  the  severity.  Little  increase  in  severity 
occurs  above  category  6  because  it  is  assumed  that  a  commander  would,  by  this  point  have 
already  reduced  training  dramatically  (13:33). 

The  range  of  categories,  0  to  9,  is  based  on  simulated  likely  combinations  of  accident 
classes  and  amounts. 

A. 2.2  Unit  Morale.  Unit  readiness  is  affected  by  unit  morale.  High  morale 
often  results  in  better  work  and  fewer  accidents;  as  unit  morale  degrades,  unit  readiness 


A-5 


1 

0B 

•h  0  £ 
U* 

<D 

> 

<D  0.4 

TS1 

02 

J 

— 

1 

1 

r 

.  1 

r 

0123456789 

Training  Execution  Category 

Figure  A. 5  Severity  Function  of  Evaluation  Measure  -  Unit  Training  Execution 

decreases.  Within  this  analysis,  unit  morale  is  made  up  of  two  parts,  lives  lost  in  unit  and 
magnitude  of  injury. 

A. 2. 2.1  Lives  Lost  in  Unit  The  severity  function  for  lives  lost  in  unit 
is  difficult  due  to  significant  differing  opinion  among  commanders.  Using  Monte  Carlo 
simulations  of  the  model,  it  was  found  that  9  fatalities  in  a  single  unit  for  100,000  Army 
flight  hours  is  extremely  rare,  but  possible.  A  few  lives  lost  in  a  unit  is  extremely  difficult, 
but  manageable.  However,  the  general  feeling  is  that  each  increasing  life  lost  would  affect 
a  unit  by  an  increasing  amount.  This  measure  evaluates  the  impact  on  morale;  it  does 
not  indicate  the  relative  importance  or  value  of  life  (13:34). 

A.  2. 2. 2  Magnitude  of  Injury.  The  measure  magnitude  of  inj  ury  approached 
unit  morale  from  the  perspective  of  the  survivors  of  an  accident,  the  soldiers  whose  lives 
were  spared.  Because  level  of  injury  is  difficult  to  quantify,  the  standard  for  evaluating 
the  category  follows: 


Table  A.l  Injury  Category  Classification 


Category  0: 
Category  1: 
Category  2: 
Category  3: 


No  injuries  requiring  hospitalization 
Injuries  requiring  <  7  days  hospitalization 

Injuries  requiring  >  7  days  hospitalization,  no  permanent  disabilities 
Injuries  resulting  in  permanent  disabilities _ 


A-6 


. . ^ 

. . . . A  _ 

* 

♦ 

•  t-H 

*-i 

<L> 

> 

0  £ 

♦ 

<D 

00 

0  .4 

♦ 

U  J.  - 

- +  . - 

♦ 

♦ 

CM 

< 

r — 

123456789 

Fatalities  in  Unit 

Figure  A.6  Severity  Function  of  Evaluation  Measure  -  Lives  Lost  in  Unit 

Severity  for  these  categories  is  based  on  the  impact  fellow  soldiers’  injuries  have 
on  unit  morale.  A  brief  stay  in  the  hospital  has  no  significant  impact  on  unit  morale. 
Slightly  more  severe  injuries  that  require  a  long  hospital  stay  may  decrease  the  morale  and 
the  severity  for  category  2  is  approximately  two  times  the  severity  of  category  1.  Finally, 
the  most  severe  injuries  are  the  ones  resulting  in  permanent  disabilities.  Soldiers  being 
injured  this  severe  on  duty  tend  to  be  remembered  by  the  remaining  soldiers  in  the  unit 
(13:35). 


Figure  A.7  Severity  Function  of  Evaluation  Measure  -  Magnitude  of  Injuries  in  Unit 
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A. 2.3  Equipment  Availability.  In  this  case,  aircraft  alone  are  the  type  of  equip¬ 
ment  that  is  addressed.  Aircraft  are  deemed  unavailable  if  an  accident  renders  them  a 
total  loss,  meaning  they  are  not  economically  repairable,  or  if  they  require  more  than  40 
hours  of  maintenance  to  repair.  Using  Monte  Carlo  simulation  and  the  ASIST  accident 
data,  the  expected  range  of  unavailable  aircraft  in  one  unit  is  0  to  3.  Because  typical 
units  may  begin  to  feel  the  most  impact  after  the  loss  of  two  aircraft,  the  largest  increase 
in  severity  occurs  between  the  unavailability  of  two  and  three  aircraft  (13:36). 


Figure  A.8  Severity  Function  of  Evaluation  Measure  -  Available  Equipment,  Aircraft 

A. 2. 4  Total  Costs.  The  investigative  team  evaluates  the  accidents  in  terms  of 
the  dollar  cost  to  the  Army.  Total  Costs  include  repair  or  replacement  of  the  damaged 
aircraft,  cost  of  injuries  and  hospitalizations,  and  damage  costs.  The  ASIST  data  enables 
the  use  of  Monte  Carlo  simulation  to  find  the  largest  possible  cost  for  100,000  Army  flight 
hours.  When  assessing  the  severity  of  dollars,  each  dollar  is  equally  important,  a  linear 
relationship  applies.  The  range  for  total  cost  is  $0  to  $99  million;  as  total  cost  increases, 
severity  increases  (13:37). 
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Figure  A. 9  Severity  Function  of  Evaluation  Measure  -  Total  Costs 
A.  3  Environmental  Damage 

Currently  accident  investigations  estimate  spillage  using  categories,  listed  below. 
The  ranges  for  the  categories  are  the  same  for  all  hazardous  fluids:  fuel,  hydraulic  fluid, 
and  oil  (9:2-11). 


Table  A. 2  Hazardous  Fluid  Spills  Classification 


Category  0: 
Category  1 
Category  2 
Category  3: 
Category  4 
Category  5 


No  hazardous  fluid  spilled 
Less  than  1  gallon  of  hazardous  fluid  spilled 
More  than  1  gallon,  but  less  than  2  gallons 
More  than  2  gallons,  but  less  than  10  gallons 
More  than  10  gallons,  but  less  than  20  gallons 
More  than  20  gallons  spilled _ 


When  evaluating  the  environmental  damage  occurring  in  100,000  flight  hours,  use 
the  category  of  the  maximum  spillage  occurrence.  This  worst-case  evaluation  estimates 
the  environmental  damage  severity  conservatively. 


A.  3.1  Soil  Damage.  Spillage  and  damage  to  the  soil  is  assessed  by  the  amount 
spilled,  using  the  previously  defined  categories.  Currently  LTC  Gleisburg  expressed  little 
difference  between  the  types  of  hazardous  fluid  spilled  into  the  soil.  Fuel,  oil,  and  hydraulic 
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Figure  A.  10  Severity  Function  of  Evaluation  Measure  -  Spills  in  Soil:  Fuel,  Oil  and 
Hydraulic  Fluid  Damage 

fluid  all  have  the  same  severity  function.  A  category  5  spill  was  estimated  to  be  3  times 
as  severe  as  a  category  4  spill,  see  Figure  A. 10  (13:39). 

A.3.2  Water  Damage.  Water  damage  requires  different  severity  functions  based 
upon  the  interaction  of  the  hazardous  fluid  and  water.  Fuel  dissipates  very  easily  in  water; 
small  quantities  are  negligible,  large  quantities  can  be  very  damaging.  Hydraulic  fluid  and 
oil  both  remain  near  the  surface  and  do  not  dissipate  in  the  water,  smaller  quantities  may 
be  slightly  more  damaging  than  small  amounts  of  fuel.  Two  separate  severity  functions 
have  been  developed.  For  evaluating  fuel  damage  in  water,  a  category  5  spill  is  10  times 
as  severe  as  a  category  4  spill,  as  described  in  Figure  A.  12.  When  evaluating  hydraulic 
fluid  and  oil  in  water,  a  category  5  spill  is  only  3  times  as  severe  as  a  category  4  spill,  as 
described  in  Figure  A.  11  (13:40). 

A. 4  Severity  of  Losses  Model  Weights 

After  obtaining  the  severity  functions,  preferences  between  the  measures  are  exam¬ 
ined  using  swing  weighting  techniques.  The  leadership  of  the  Safety  Center  that  evaluated 
the  Severity  of  Losses  placed  most  importance  on  casualties  and  secondly  on  unit  readi¬ 
ness.  Specifically,  fatalities  contribute  the  most  to  the  severity  of  a  block  of  accidents. 
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Figure  A.  11  Severity  Function  of  Evaluation  Measure  -  Spills  in  Water:  Oil  and  Hy¬ 

draulic  Fluid  Damage 

Accidents  lowering  the  training  execution  and  consequently  reducing  unit  readiness  was 
weighted  second  most  heavily.  Note  the  minimal  weight  on  environmental  damage.  This 
objective  is  included  because  it  is  very  important  to  the  Army;  however,  at  this  point 
in  time,  the  Safety  Center  does  not  have  the  data  available  to  accurately  account  for  its 
contribution. 

For  a  more  detailed  description  of  both  the  severity  functions  and  the  elicited  weights, 
reference  Gallan,  2000  or  Sperling,  1999. 
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Figure  A.  12  Severity  Function  of  Evaluation  Measure  -  Spill  in  Water:  Fuel  Damage 


Figure  A.  13  Severity  of  Losses  Global  Weights 
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Appendix  B.  Severity  Reduction  Simulation 

In  order  to  follow  the  code,  it  is  first  necessary  to  understand  the  basic  framework  behind 
what  is  occurring.  This  appendix  with  begin  with  a  general  overview  of  what  occurs.  It 
will  follow  with  an  in-depth  explanation  of  how  the  code  executes  this  framework,  including 
references,  by  name,  to  the  macros  and  functions  used. 

For  each  portfolio  to  be  analyzed,  the  {0,  1}  portfolio  of  controls  model  is  exam¬ 
ined.  It  finds  the  hazards  reduced  by  these  controls  and  creates  hazard  and  effectiveness 
lists.  These  fists  are  inputs  to  the  main  macro  that  drives  each  iteration.  The  itera¬ 
tions  randomly  select  a  certain  number  of  possible  accidents  to  occur.  The  accidents  are 
then  randomly  drawn  from  the  accident  database.  Because  each  accident  has  associated 
hazards  that  cause  the  accident  or  increase  its  severity,  by  applying  controls  that  decrease 
the  severity,  we  see  can  gain  an  approximate  decrease  in  severity  that  the  controls  supply. 
The  severity  reduction  is  recorded  for  each  iteration.  The  multiple  runs  provide  a  data 
set  from  which  the  expected  value  of  a  portfolio  can  be  obtained.  The  expected  values 
become  the  inputs  for  the  direct  search  optimization  procedure.  Figure  B.l  represents  the 
functional  flow  of  calculations  within  the  simulation. 


Figure  B.l  An  Overview  of  the  Severity  Reduction  Simulation 
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Figure  B.2  Function  Single  Run  Input  and  Output 


j 3.0.1  A  single  iteration .  Prior  to  beginning  any  simulation,  the  following  data 
must  be  located  in  worksheets  within  in  Army_FMA_Data.xls: 

1.  Normalized  hazard  contributions  in  “NormAnalysisGroupData” 

2.  Accident  investigation  by  case  number  (all  types  of  aircraft)  in  “AccidentData” 

3.  Evaluation  measure  ranges  and  functions  in  “EvalMeasures  Values” 

The  individual  hazard  assessments  for  each  aircraft  are  divided  by  aircraft  type  into 
the  worksheets  AH64_Data,  H47_Data,  OH58D_Data,  and  UH60_Data.  Within  each 
worksheet,  matrices  contain  hazard  assessments,  by  case  number,  for  Casualties  (1),  Cost 
(2),  Frequency  (3),  and  Prevention  (4).  Individual  data  matrices  for  each  aircraft  contain 
three  dimensions:  case  number,  hazard  identifier,  and  the  number  1  through  4  representing 
the  type  of  hazard  assessment,  as  noted  above.  This  data  as  generated  by  the  accident 
investigation  teams  and  evaluated  by  ASIST  is  the  foundation  for  this  entire  study.  For 
reference,  it  is  important  to  note  that  the  H47  information  is  notional  and  should  be 
updated  before  any  decisions  are  based  on  this  study. 

The  function  Single_Run  acts  as  the  parent  function  called  by  @RISK  and  itself 
calling  the  remaining  functions  and  subroutines.  Ultimately,  the  objective  of  this  function 
is  to  take  in  the  distribution  of  the  possible  accident  rate  and  to  produce  a  90%  confidence 
interval  for  the  range  of  reduction  possible  by  using  the  specified  portfolio. 

Single  Run  calls  the  subroutine  Get_Haz_and_Eff  to  obtain  the  hazards  and  ef¬ 
fectiveness  of  the  portfolio  of  controls. 

Single_Run  also  calls  the  function  Do_A_Rep  which  returns  the  average  percent 
severity  reduction  of  a  set  of  runs  representing  26  blocks  of  100,000  flight  hours,  or  five 
years  of  FMA  flight. 
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HazListx={  AH64-25,. . . 
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Figure  B.3  Subroutine  Get  Hazards  and  Effectiveness  Input  and  Output 
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Figure  B.4  Function  Do  A  Replication  Input  and  Output 


Function  Do_A_Rep  calls  Fill_Matrices_and_Arrays  which  takes  the  given  air¬ 
craft  data  from  the  individual  workbooks  and  stores  all  of  the  data  in  dimensioned  matrix 
form  in  the  program’s  memory.  This  procedure  increases  the  speed  of  processing  informa¬ 
tion.  Fill_Matrices_and_Arrays  is  only  called  for  the  first  iteration  of  Do_A_Rep;  all 
subsequent  iterations  recognize  the  data  in  memory  and  proceed  with  calculations. 

Do_A_Rep  performs  iterations  obtaining  the  percent  severity  reduction  for  single 
observations  of  blocks  representing  100,000  flight  hours.  The  force  modernized  aircraft 
addressed  by  this  study  flew  over  550,000  flight  hours  in  FY2000.  The  number  of  hours 
flown  during  FY94  through  FY98,  sum  to  over  2,500,000  flight  hours.  By  simulating  26 
blocks  of  100,000  flight  hours,  a  mean  severity  reduction  for  the  5  observed  years  can  be 
obtained.  In  order  to  simulate  these  blocks,  Rand_draw  must  be  called. 

The  objective  of  this  function  is  to  take  a  random  sample  of  possible  accidents  and 
test  the  improvement  gained  by  applying  a  certain  portfolio  of  controls.  Of  all  possible 
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Figure  B.5  Subroutine  Fill  Matrices  and  Arrays  Input  and  Output 
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Figure  B.6  Function  Random  Draw  Input  and  Output 


accidents,  the  specified  number  are  selected,  the  effectiveness  applied  and  the  severity  score 
calculated  with  and  without  new  controls.  The  percent  difference  between  these  severity 
scores  is  the  value  returned  to  Do_A_Rep. 

Do_A_Rep  takes  the  single  severity  reduction  observations  and  averages  them  to 
obtain  a  value,  the  mean  severity  reduction,  to  return  to  Single_Run.  The  value  obtained 
from  this  simulation  is  used  by  the  optimization. 

This  spreadsheet  tool  has  user-friendly  macros  that  will  enable  the  sponsor  to  format 
their  data  an  use  this  simulation  technique  in  the  future. 
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Appendix  C.  Control  Lists  for  Force- Modernized  Aircraft 
The  following  controls  were  defined  by  ASIST  to  reduce  hazards  for  the  following  aircraft. 


Table  C.l  Abbrevations  for  Control  Lists 


Abbreviation 

DTLOMS  Area 

org 

organization 

trg 

training 

ldr  dev 

leadership  development 

doct 

doctrine 

C.l  Control  List  for  the  AH-  64  Apache 


CONTROL 

CONTROL  STATEMENT 

DTLOMS 

COST 

AH64-C01 

Modify  force  trim  switch  (no  off  position). 

Materiel 

Low 

AH64-C02 

Amend  -10  warning. 

Materiel 

Nil 

AH64-C03 

Retrofit  ECP0887  (cyclic  bellcrank  counterweights). 

Materiel 

Low 

AH64-C04 

Mechanical  Stop  tied  to  Squat  Switch  (operative  dur¬ 
ing  shutdown) 

Materiel 

Medium 

AH64-C05 

Improved  flight  control  system  (ACAH+  Hover  hold). 

Materiel 

Medium 

AH64-C06 

Second  generation  FLIR. 

Materiel 

High 

AH64-C07 

Obstacle/ wire/ground  proximity  warning  device. 

Materiel 

High 

AH64-C08 

”  Family  of  Virtual  Rotor  Disk/Prox  Warning  System” 

Materiel 

AH64-C09 

Revisit  external  stores  jettison  protocols  and  restric¬ 
tions 

Doctrine 

VLow 

AH64-C10 

Develop,  monitor  and  evaluate  a  Crew  Coordination 

Sustainment  Training  program  integrated  into  all  avi¬ 
ation  tasks 

Training 

VLow 

AH64-C11 

Standardized  multi-aircraft  mission  abort 

Training 

VLow 

AH64-C12 

Hands  on  inadvertent  IMC  Training  either  in  the  sim¬ 
ulator  or  aircraft 

Training 

VLow 

AH64-C13 

Pre-brief  on  inadvertent  IMC  contigency 

Doctrine 

High 

AH64-C14 

Increase  Flying  Hour  Program 

Org 

VLow 
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CONTROL 

CONTROL  STATEMENT 

DTLOMS 

COST 

AH64-C15 

Improve  FLIR  condition  forecasting  techniques 

Materiel 

VLow 

AH64-C16 

Enhance  risk  management  training 

Training 

Low 

AH64-C17 

Envelope  cueing  (exceedances). 

Materiel 

Medium 

AH64-C18 

Digital  source  collector  and  develop  procedures  for  use 

of  FDR  data  by  commanders  for  aircrew  training 

Materiel 

Medium 

AH64-C19 

Develop  tools  to  help  commanders  identify  high  risk 

behavior 

Ldr  Dev 

AH64-C20 

Modifying  flight  symbology  (VSI) 

Materiel 

Low 

AH64-C21 

Digital  Source  Collector  (DSC)  and  envelope  cueing 

(exceedences)  and  notice  to  pilot  of  exceedences/crew 

monitor 

Materiel 

High 

AH64-C22 

Improve  PNVS 

Materiel 

High 

AH64-C23 

Flight  Symbology  (velocity  vector  &  VSI)  overlay  on 

TADS 

Materiel 

Low 

AH64-C24 

Standards  or  guidance  for  risk  assessment  articulation 

for  cummulative  risk 

Doctrine 

VLow 

AH64-C25 

Make  an  evaluation  task  (hovering  in  close  prox  to 

terrain,  degraded  visual  environment,  and  high  work¬ 
load)***  DES  Clarification 

Training 

VLow 

AH64-C26 

Family  of  controls  for  vision  enhancement  systems 

Materiel 

AH64-C27 

Integrate  improved  night  vision  goggle  system  and 

training  for  CPG 

Materiel 

Low 

AH64-C28 

Fence  existing  flying  hour  dollars 

Training 

Nil 

AH64-C29 

Standardize  use  of  ERFS  (restrict  for  theater  self  de¬ 
ployment  -  including  train  ups  -  or  ferry  missions  only) 

through  doctrinal  change 

Doctrine 

Nil 

AH64-C30 

Acquire  crashworthy  ERFS 

Materiel 

AH64-C31 

Communicate  HQDA  acceptance  of  risk  for  current  use 

of  ERFS 

Org 

Nil 
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CONTROL 

CONTROL  STATEMENT 

DTLOMS 

COST 

AH64-C32 

Install  fire  detection  and  suppression  in  T-back  area 

to  include  the  forward  area 

Materiel 

Medium 

AH64-C33 

Install  new  APU  clutch  w/  disengage  feature  (one¬ 
way). 

Materiel 

Medium 

AH64-C34 

Develop  new  anti-flail  containment  for  APU  drive 

shaft. 

Materiel 

Low 

AH64-C35 

Integrate  new  anti-flail  containment  for  APU  drive 

shaft  into  Drive  train  2000 

Materiel 

Nil 

AH64-C36 

Integrate  new  APU  clutch  w/  disengage  feature  (one¬ 
way)  into  Drive  train  2000 

Materiel 

Low 

AH64-C37 

Redesign  APU  clutch 

Materiel 

Medium 

AH64-C38 

Mandatory  replacement  with  new  yellow  harness 

Materiel 

VLow 

AH64-C39 

Establish  requirement  for  emergency  procedure  train¬ 
ing  for  single  engine  failure  in  the  Combat  Mission 

Simulator  (Increase  awareness  of  high  side  or  low  side 

failures  ) 

Training 

Nil 

AH64-C40 

Increase  Sz  improve  data  captured  for  Class  C  acci¬ 
dents  through  the  use  of  DA  2397  forms 

Org 

VLow 

AH64-C41 

Shroud  tail-rotor 

Materiel 

High 

AH64-C42 

Standardize  doctrine  and  operational  procedures  for 

mulit-ship  operations  involving  mixed  night  vision  sys¬ 
tems,  including  associated  training  (flight  and  simula¬ 
tor) 

Doctrine 

Medium 

AH64-C43 

Defer  to  USAALS  to  look  at  maintenance  related  con¬ 
trols  to  reduce  FOD  hazards  associated  with  tools 

Org 

VLow 

AH64-C44 

Defer  to  USAALS  to  look  at  maintenance  related  con¬ 
trols  to  reduce  hazards  associated  with  maintenance 

discipline 

Org 

VLow 

C-3 


CONTROL 

CONTROL  STATEMENT 

DTLOMS 

COST 

AH64-C45 

Increase  damage  tolerance  of  aircraft 

Materiel 

High 

AH64-C46 

Emphase  proper  pilot  reaction  to  impending  bird 

strike  (maintain  steady  flight) 

Training 

VLow 

AH64-C47 

Cockpit  indication  of  component  security 

Materiel 

Medium 

AH64-C48 

Value  engineering  study  of  fasteners 

Materiel 

VLow 

AH64-C49 

Safety  Latch  (mechanical  secondary  latch  for  the  tail 

rotor  driveshaft  cover) 

Materiel 

Low 

AH64-C50 

Establish  improved  technique  (as  part  of  preflight)  to 

physically  ensure  integrity  of  cowlings 

Training 

Nil 

AH64-C51 

Redesign/upgrade  of  latching  devices 

Materiel 

Low 

AH64-C52 

Require  correct  size  cam  locks  be  installed  in  each  ap¬ 
plication 

Training 

Nil 

AH64-C53 

USAALS  evaluate  possible  elimination  of  multiple  size 

cam  locks 

Materiel 

VLow 

AH64-C54 

Paint  leading  edge  of  engine  cowling  day  glow  orange 

Materiel 

Nil 

AH64-C55 

Book  of  Hazards  and  Controls 

Training 

VLow 

AH64-C56 

Establish  a  command  information  system  which  tracks 

all  forms  of  high  risk  behavior  and  marginal  perfor¬ 
mance 

Materiel 

Low 

AH64-C57 

Modify  AR  95-3  to  require  that  mission  planning  time 

is  considered  as  mandatory  topic  for  risk  determina¬ 
tion  and  establish  risk  management  standards  for  mis¬ 
sion  planning  time 

Ldr  Dev 

Nil 

AH64-C58 

Integrate  risk  management  training  into  aviation  of¬ 
ficer/ WO/NCO  development  programs  (use  accident 

experience  as  part  of  the  training) 

Org 

Low 

AH64-C59 

Develop  a  standardized  methodology  for  conducting 

mission  risk  assessments  with  the  objective  for  iden¬ 
tification  of  all  hazards  and  associated  controls  to  be 

presented  in  the  mission  brief 

Training 

VLow 
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CONTROL 

CONTROL  STATEMENT 

DTLOMS 

COST 

AH64-C60 

Establish  model  RM  training  program,  starting  with 

DAIG  evaluation  of  all  institutional  training  schools 

for  integration  of  risk  management  training  into  cur¬ 
riculum 

Training 

VLow 

AH64.C61 

Establish  minimum  training  requirements  (concerning 

seleced  enforcement  of  aviation  maintenance  and  oper¬ 
ations  standards)  for  assignment  as  commanders  (in¬ 
tegration  into  advanced  course,  an  exportable  training 

package,  and  modifying  BOC  and  AOC) 

Training 

VLow 

AH64-C62 

Establish  or  enforce  selection  criteria  for  advanced  avi¬ 
ator  training 

Training 

VLow 

AH64-C63 

Increase  command  emphasis  (Advance  Course)  on 

safety  incentives 

Training 

VLow 

AH64-C64 

Evaluate  AH-64  maintenance  force  structure. 

Org 

VLow 

AH64-C65 

Provide  an  effective  back-up  control  system  (no  tran¬ 
sients,  redundancy,  transparent  -equivalent  control  feel 

and  response) 

Materiel 

High 

AH64-C66 

Reactivate  BUCS 

Materiel 

Medium 

AH64-C67 

Remove  BUCS  (Consider  Red  Team  for  mishap  data 

and  effectiveness  rating) 

Materiel 

Low 

AH64-C68 

Redesign  primary  flight  control  system  to  military 

specification  (redundancy,  ballistic  tolerance,  and 

strength  requirement) 

Materiel 

High 

AH64-C69 

Reactivate  BUCS  in  pre  PV  529  aircraft  and  AH64D 

lot  1  a/c 

Materiel 

Low 

AH64-C70 

Re-orient  direction  of  swage  pin 

Materiel 

VLow 

AH64-C71 

Redesign  tail  rotor  control  to  provide  fixed  medium 

pitch  setting  equivalent  to  nominal  thrust  to  transition 

to  controlled  flight  and  controlled  landing  (rewrite  -10 

for  new  emergency  procedure) 

Materiel 

Medium 

C-5 


CONTROL 

CONTROL  STATEMENT 

DTLOMS 

COST 

AH64-C72 

Install  a  new  design  strap  pack  with  an  established 

Materiel 

Medium 

service  life 

AH64-C73 

Revise  and  enforce  government  oversight  procedures 

Materiel 

VLow 

(AR  95-20) 

AH64-C74 

Revise  -23  with  new  NDI  inspection  and  procedures 

Materiel 

VLow 

AH64-C75 

Redesign  refuel  nozzle  to  include  visual  locking  indi¬ 

Materiel 

Low 

cation 

AH64-C76 

Increase  emphasis  on  assembly  of  refueling  equipment 

Training 

Low 

and  hot  refueling  during  POL  operator  training  (initial 
training  and  new  requirement  for  recurring  qualifica¬ 
tion  training  and  annual  recertification) 


AH64-C77 

Provide  Personal  Protective  Equipment  to  POL  han¬ 
dlers 

Materiel 

Low 

AH64-C78 

Install  cockpit  airbag  system 

Materiel 

High 

AH64-C79 

Install  MA-16  inertia  reels 

Materiel 

VLow 

AH64-C80 

Cooler  search  light  bulb 

Materiel 

Low 

AH64-C81 

Install  heat  shield  for  search  light 

Materiel 

Low 

AH64-C82 

Add  a  light  on  the  caution  advisory  panel  to  indicate 

search  light  condition  (on/off) 

Materiel 

Low 

AH64-C83 

Install  Flight  Data  Recorder 

Materiel 

Medium 

AH64-C84 

Pressurize  cockpit  to  redirect  airflow  out  of  the  cockpit 

Materiel 

Medium 

AH64-C86 

Redesign  internal  fuel  system  (baffled  system) 

Materiel 

Medium 

AH64-C87 

Install  air  tolerant  fuel  pump  (bubble  eating  pump) 

Materiel 

Medium 

AH64-C88 

Redesign  fuel  transfer  system  to  a  fuel  suction  system 

Materiel 

Medium 

AH64-C89 

Install  automatic  external  fuel  transfer  shutoff  prior  to 

ingestion  of  air 

Materiel 

Medium 

AH64-C90 

Install  automatic  fuel  management  system  (including 

management  of  aux  fuel) 

Materiel 

Medium 

AH64-C92 

Obstacle  detection  system 

Materiel 

Medium 
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CONTROL 

CONTROL  STATEMENT 

DTLOMS 

COST 

AH64-C94 

Establish  lot  accept  testing  controls  for  aircraft  ord¬ 
nance 

Materiel 

Low 

AH64-C95 

Command  emphasis  campaign  from  HQDA(CSA), 

Aviation  Branch  Chief,  and  Aviation  principals  to  en¬ 
force  standards 

Ldr  Dev 

VLow 

AH64-C96 

Study  and  establish  minimum  operational  experience 

and  flight  time  requirements  for  selection  as  aviation 

commander 

Ldr  Dev 

Low 

AH64-C97  Provide  training  through  Pre-command  Course  to 

increase  awareness  of  need  for  Bn  Commander  to 

broaden  junior  officer  development  -  management  of 

flying  hour  program  and  risk  management 

C.2  Control  List  for  the  Chinook 

Ldr  Dev 

VLow 

CONTROL 

CONTROL  STATEMENT 

DTLOMS 

COST 

H47-C01 

Enhanced  risk  management  training  at  all  levels  of 

command 

Training 

VLow 

H47-C02 

Obstacle  collision  warning  system  which  includes  pro¬ 
visions  for  Pi-adjustable  parameters. 

Materiel 

Medium 

H47-C03 

Increase  pilot  flight  hours  and  supervision 

Training 

Medium 

H47-C04 

Digital  source  collector  and  develop  procedure  for  use 

of  DSC  data  by  commanders  for  aircrew  training. 

Materiel 

Medium 

H47-C05 

WSPS 

Materiel 

Medium 

H47-C06 

Crew  coordination  sustainment  training 

Training 

Low 

H47-C07 

Back-up  DC  ( battery )-powered  IMC  instruments 

Materiel 

Low 

H47-C08 

Install  state-of-the-art  waterproof  circuit  breaker 

Materiel 

Low 

H47-C09 

Seal  fuselage  to  prevent  water  intrusion 

Materiel 

VLow 

H47-C10 

Redesign  fuselage  to  prevent  water  intrusion 

Materiel 

High 
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CONTROL 

CONTROL  STATEMENT 

DTLOMS 

COST 

H47-C11 

Increase  manning  levels  to  match  present  operations  in 

aviation  units,  considering  non-aviation  requirements 

Soldiers 

High 

H47-C12 

Develop  and  provide  training  to  commanders  to  match 

crew  experience  with  mission  requirements  to  include 

risk  management  training 

Training 

Low 

H47-C13 

Reverify  240-23  flight  control  hardware  installation 

against  H-47D  production  drawings 

Materiel 

VLow 

H47-C14 

Training  for  CCAD  personnel  in  CH-47-peculiar  main¬ 
tenance  practices 

Materiel 

VLow 

H47-C15 

Increase  technical  oversight  of  the  maintenance  manual 

(increase  personnel) 

Org 

Low 

H47-C16 

Increase  PEO/PM/AMCOM  system  dedicated  field 

representation  to  provide  technical  oversight  and  op¬ 
erational  feedback. 

Org 

Low 

H47-C17 

Add  ground  proximity  (squat)  switches  to  forward 

landing  gear  which  limits  responsiveness  of  control  in¬ 
put 

Materiel 

Low 

H47-C18 

add  warning  to  dash  10  for  limits  on  upslope  landing 

to  include  adding  forward  cyclic  while  on  upslope 

Training 

Nil 

H47-C19 

Develop  risk  assessment  procedure  for  waivers  to  AR 

95-1  requirements.  Process  Issue:  Need  to  change 

waiver  process  into  a  risk  management  process 

Doctrine 

VLow 

H47-C20 

Standardize  cargo  release  operations,  with  a  change  to 

the  ATM,  mandating  use  of  hot  mike  during  external 

operations  whenever  master  cargo  hook  is  activated. 

Training 

Nil 

H47-C21 

improved  design  of  cargo  release  switch 

Materiel 

Low 

H47-C22 

Modify  AR  95-1  to  mandate  seatbelt  usage  when  not 

performing  flight  crew  duties,  (may  apply  to  other 

Training 

Nil 

aircraft) 
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CONTROL 

CONTROL  STATEMENT 

DTLOMS 

COST 

H47-C23 

Change  phase  maintenance  manual  requiring  bench 

test  of  AFCS  computer  at  phase. 

Materiel 

VLow 

H47-C24 

Modify  -23  to  establish  TBO  for  integrated  lower  con¬ 
trol  actuators  (ILCAs)  and  review  adequacy  of  upper 

boost  actuators  (UBAs)  TBO 

Materiel 

Low 

H47-C25 

Develop  emergency  procedure  for  flight  control  lockup 

Training 

VLow 

H47-C26 

Modify  flight  control  system  to  provide  inflight  indi¬ 
cation  of  Integrated  Lower  Control  Actuator  (ILCA) 

and  Upper  Boost  Actuator  (UBA)  jams 

Materiel 

High 

H47-C27 

Develop  and  implement  new  Army  Oil  Analysis  Pro¬ 
gram  (AOAP)  procedure  for  hydraulic  fluid  analysis 

Materiel 

Low 

H47-C28 

Develop  new  standards  to  increase  number  of  fastrope 

crew  members  for  missions 

Org 

Nil 

H47-C29 

Redesign  HAR  refeuling  probe 

Materiel 

High 

H47-C30 

Change  SOP  responsibility  for  search  light  controls  to 

flight  engineer 

Training 

Nil 

H47-C31 

Develop  and  train  HAR  refueling  scenerio  in  a  high 

fidelity  simulatior 

Mat  /Trg 

Low 

H47-C32 

Improve  tanker  cueing 

Materiel 

Medium 

H47-C33 

Install  day  and  night  remote  viewing  devices  to  view 

cargo  hook  operations 

Materiel 

Low 

H47-C34 

4  axis  hover  hold 

Materiel 

Medium 

H47-C35 

Install  winchable  hook  assemblies 

Materiel 

High 

H47-C36 

Modify  -10  to  mandate  4th  crew  member  for  sling  load 

operations 

Soldiers 

Low 

H47-C37 

automated  approach  landing  system  (improved  FCS) 

Materiel 

Medium 

H47-C38 

Information  system  to  provide  realtime  weather  at  unit 

operations 

Materiel 

VLow 

H47-C39 

Second  generation  FLIR 

Materiel 

High 
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CONTROL 

CONTROL  STATEMENT 

DTLOMS 

COST 

H47-C40 

update  ATM  TC1-216  to  include  landing  procedures 

for  various  terrain 

Training 

Nil 

H47-C41 

Mandate  a  regulatory  requirement  to  ensure  airfields 

are  marked  IAW  TM  5-803-4  and  deviations  published 

in  FLIP. 

Doctrine 

Nil 

H47-C42 

Modify  appropriate  TM  to  require  marking  of  all  po¬ 
tential  hazards/obstacles. 

Doctrine 

Nil 

H47-C43 

Modify  MOS  program  for  67U  to  include  aca¬ 
demic/practical  training  for  ground  guide/ wing  walker 

responsibilities. 

Training 

VLow 

H47-C44 

Pre-mission  briefing  of  all  expected  hazards  and  con¬ 
trols  to  be  used. 

Doctrine 

Nil 

H47-C45 

audio  volume  control  on  low  altitude  warning 

Materiel 

VLow 

H47-C46 

terrain  avoidance  radar  (coupled  into  FCS) 

Materiel 

High 

H47-C47 

Reinstitute  ATM  Task  1078  (Unusual  Attitude  Recov¬ 
ery)  for  VMC 

Materiel 

High 

H47-C48 

Develop  an  emergency  flight  control  response  limiter 

Materiel 

High 

H47-C49 

improved  NVG 

Materiel 

Low 

H47-C50 

Increase  information  recorded  on  accident  forms,  use 

DA-2397  series  forms. 

Doctrine 

VLow 

H47-C51 

Command  emphasis 

Ldr  Dev 

Nil 

H47-C52 

Develop  a  better  debris  detection  system  using  "full- 

flow”  technology. 

Materiel 

Low 

H47-C53 

Develop  a  mean  or  reporting  deployability  status  of 

mission  equipment  (ERFS) 

Training 

VLow 

H47-C54 

Change  location  of  N2  speed  feedback  assembly 

Materiel 

Medium 

H47-C55 

Accident  scenarios  in  simulator 

Training 

Nil 

H47-C56 

Design  provisions  for  protective  stowage  of  the  cargo 

release  switch 

Materiel 

Low 
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CONTROL 

CONTROL  STATEMENT 

DTLOMS 

COST 

H47-C57 

Install  DSC  in  all  H-47 

Materiel 

Medium 

H47-C58 

Improve  existing  non-slip  surface 

Materiel 

Nil 

H47-C59 

’’dead  man”  ramp  control  switch 

Materiel 

VLow 

H47-C60 

Accelerate  installation  of  ECP  D229 

Materiel 

Low 

H47-C61 

accelerate  installation  of  MWO  55-1520-240-50-52 

combat  lighting 

Materiel 

Low 

H47-C62 

accelerate  installation  of  the  collector  gear  cartridge 

ECP 

Materiel 

Medium 

H47-C63 

Accelerate  installation  of  the  older  style  (-5)  fan  shaft 

Materiel 

Medium 

H47-C64 

ASAM-99-02 

Materiel 

Nil 

H47-C65 

Better  troubleshooting  procedures 

Training 

VLow 

H47-C66 

Clarify  SOPs  (Ranger  vs.  160th) 

Training 

Nil 

H47-C67 

Clear  crew  coordination 

Training 

Nil 

H47-C68 

Combat  engineers  survey  and  maintain  areas  for  flight 

operations 

Org 

High 

H47-C69 

Communication  plan  (Ship  to  ground  as  well  as  ship 

to  ship) 

Training 

Nil 

H47-C70 

Complete  with  an  adequate  MWO  to  address  the  spirit 

of  the  Boeing  bulletin 

Materiel 

Low 

H47-C72 

Crew  and  Pax  briefings 

Training 

Nil 

H47-C73 

cross  training  of  Crewchiefs  and  Fast-Rope  Safeties 

Training 

Nil 

H47-C74 

Develop  a  quick  and  easy  reference  to  develop  LZ  sizes 

for  diferent  aircraft 

Training 

Nil 

H47-C75 

Develop  a  wear  indicator  gage 

Materiel 

Low 

H47-C76 

Develop  crew  member  emergency  procedure  training 

in  simulator 

Training 

VLow 

H47-C77 

Develop  FE/CE  requirement  in  TC  1-216  to  obtain  PI 

Training 

Nil 

clearance  to  lower  ramp  after  landing. 
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CONTROL 

CONTROL  STATEMENT 

DTLOMS 

COST 

H47-C78 

Develop  policies  and  procedures  for  weather  risk  man¬ 
agement  decisions  through  unit  operations  center  (to 

include  maintenance  operations) 

Training 

VLow 

H47-C79 

Develop  procedure  to  track  maintenance  trends 

Doctrine 

Nil 

H47-C80 

Emergency  procedure  standardization  during  mainte¬ 
nance  test  flights. 

Training 

Nil 

H47-C81 

Enhance  communication  between  Fast  Rope  Master 

and  Safety 

Training 

Nil 

H47-C82 

Enhance  crew  coordination  (call  out  for  arming  and 

alert  caution  to  release  button)  with  caution  box  in 

the  ATM 

Materiel 

Nil 

H47-C83 

Ensure  chip  detector  design  and  location  is  appropriate 

to  indicate  presence  of  metal  particles  in  the  oil 

Materiel 

Low 

H47-C84 

Establish  a  maintenance  FE  position  for  maintenance 

test  flights 

Soldiers 

Medium 

H47-C85 

Establish  overhaul  interval  for  APU 

Materiel 

Medium 

H47-C86 

Etching  on  the  striker  plate  to  provide  indication  of 

wear 

Materiel 

Low 

H47-C87 

Folding  rotor  system  (SH-60) 

Materiel 

High 

H47-C89 

harnesses  for  flight  engineers  to  help  with  leverage 

Materiel 

Medium 

H47-C90 

Highlight  striker  plate  wear  as  an  emphasis  item  in 

preflight 

Training 

Nil 

H47-C91 

Identify  and  track  special  classification  for  potentially 

hazardous  conditions 

Training 

N/A 

H47-C92 

Improve  avionics  door  latch 

Materiel 

Low 

H47-C93 

Improve  crew  coordination/planning  for  emergency 

situations 

Training 

Nil 
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CONTROL 

CONTROL  STATEMENT 

DTLOMS 

H47-C94 

Improve  handling  characteristics  at  high  speed  flight 

by  providing  means  to  fine  tune  speed  management  or 

improve  AFCS 

Materiel 

H47-C95 

Improve  NVD  (field  of  view  and  visual  acuity)  to  help 

identify  closure  rates 

Materiel 

H47-C96 

Improve  understanding  and  confidence  in  navigaion 

equipment 

Training 

H47-C97 

improve  winch  to  perform  ingress  assistance  of  indi¬ 
vidual 

Materiel 

H47-C99 

Incorporate  tools  in  the  Risk  Management  process  for 

accountability  of  the  increased  risk  due  to  lack  of  fund¬ 
ing. 

Ldr  Dev 

H47-C100 

Increase  available  funding  for  safety  ECPs 

Materiel 

H47-C101 

Increase  awareness  on  the  ground  commander  to  an¬ 
ticipate  requirements  for  LZ 

Training 

H47-C102 

Increase  emphasis  on  emergency  procedures 

Training 

H47-C103 

Increase  oil  capacity 

Materiel 

H47-C104 

Information  system  to  provide  realtime  weather  at  unit 

operations 

Materiel 

H47-C105 

Install  a  backup  oil  cooling  system 

Materiel 

H47-C106 

Install  containment  ring  around  APU 

Materiel 

H47-C107 

Install  DSC  and  HUMS  to  capture  data 

Materiel 

H47-C108 

Onboard  weather  avoidance  equipment  to  facilitate 

identification  of  storm  cells  during  flight 

Materiel 

H47-C109 

Install  DSC  to  capture  data  to  use  in  clutch  perfor¬ 
mance  and  TBO  determination  analysis 

Materiel 

H47-C110 

Install  FADEC 

Materiel 

H47-C111 

Install  HUMS 

Materiel 

COST 

High 

High 

Nil 

Medium 

Nil 

High 

Nil 

Nil 

Medium 

Low 

Medium 

Low 

Medium 

Low 

High 

High 

Medium 
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CONTROL 

CONTROL  STATEMENT 

DTLOMS 

COST 

H47-C112 

Install  increased  de-icing  capability  on  all  H-47s 

Materiel 

High 

H47-C113 

Install  new  compressor  wheels  in  APU 

Materiel 

Low 

H47-C114 

Limit  number  of  winch  lift  cycles  to  minimize  injury 

risk 

Training 

Nil 

H47-C115 

Load  sensor  to  detect  load  on  ground  to  allow  crew 

chief  to  release  load  (this  control  also  requires  move¬ 
ment  of  emergency  release  button  and  enhanced  emer¬ 
gency  procedures) 

Materiel 

High 

H47-C116 

mandate  use  of  metal  clevis  to  reach  pendant 

Materiel 

Nil 

H47-C117 

Mandatory  retrofit  of  -11  lag  dampener 

Materiel 

Low 

H47-C118 

Match  personnel  physical  standards  with  task  require¬ 
ments 

Organization 

Nil 

H47-C119 

Match  personnel  to  mission  requirement  according  to 

SOP 

Soldiers 

Nil 

H47-C120 

Modify  aircraft  to  provide  greater  crew  visibility  di¬ 
rectly  beneath  the  aircraft 

Materiel 

Low 

H47-C121 

Night  Vision  Devices  for  FAST  Rope  Masters  and 

Safeties 

Training 

Nil 

H47-C122 

Provide  aircrew  training  on  ground  resonance 

Training 

Nil 

H47-C123 

Provide  standardized  configuration  for  securing 

winch/hoist  control  grip  assembly  during  external  load 

operations 

Training 

Nil 

H47-C124 

reconfigure  load  (remove  blades) 

Training 

Nil 

H47-C125 

Redesign  engine  control  system  to  prevent  rotor  over¬ 
speed. 

Materiel 

High 
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CONTROL 

CONTROL  STATEMENT 

DTLOMS 

COST 

H47-C126 

Redesign  latch  to  meet  operational  loads  and  vibra¬ 
tions 

Materiel 

Low 

H47-C127 

Redesign  the  transmission  and  oil  cooling  system 

Materiel 

High 

H47-C129 

Remote  wireless  ramp  control 

Materiel 

Medium 

H47-C130 

Research  new  procedures  and  methods  for  flight  con¬ 
trol  hydraulic  system  inspections/maintenance  on  re¬ 
curring  basis 

Materiel 

Low 

H47-C131 

Operational  Risk  Management  enhancements 

Training 

VLow 

H47-C132 

Safety-of-Flight  to  remove  APUs  past  xxx  hrs. 

Materiel 

Medium 

H47-C133 

Separate  funding  from  the  decision  making  process 

Materiel 

Nil 

H47-C134 

Smart  drogue  to  detect  fuel  line  tension  and  safely 

maintain  engagement 

Materiel 

Medium 

H47-C136 

Symposium  or  other  forums  with  USAF/USA  to  high¬ 
light  division  of  tasks/hazards/risks/controls  associ¬ 
ated  with  HAR 

Training 

VLow 

H47-C137 

Update  TC  1-216  to  focus  on  crew  member  communi¬ 
cation  skills 

Training 

Nil 

H47-C138 

use  of  high  temperature  tolerant  materials 

Materiel 

N/A 

H47-C139 

voice  activated  communication  system 

Materiel 

Medium 

H47-C140 

Provide  training  through  Pre-command  Course  to 

increase  awareness  of  need  for  Bn  Commander  to 

broaden  junior  officer  development  -  management  of 

flying  hour  program  and  risk  management 

Ldr  Dev 

VLow 

H47-C141 

Study  and  establish  minimum  operational  experience 

and  flight  time  requirements  for  selection  as  aviation 

commander 

Ldr  Dev 

Low 

H47-C142 

Command  emphasis  campaign  from  HQDA(CSA), 

Ldr  Dev 

VLow 

Aviation  Branch  Chief,  and  Aviation  principals  to  en¬ 
force  standards 
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C.3  Control  List  for  the  OH-58D  Kiowa  Warrior 


CONTROL 

OH58-C01 

OH58-C02 

OH58-C03 

OH58-C04 

OH58-C05 

OH58-C06 


OH58-C07 

OH58-C08 

OH58-C09 

OH58-C10 

OH58-C11 


OH58-C12 


OH58-C13 

OH58-C14 

OH58-C15 


CONTROL  STATEMENT 

DTLOMS 

COST 

Relocate  CSC  to  improve  visibility/accessibility 

Materiel 

Low 

Field  the  improved  CSC  (voice  activated)  throughout 

the  fleet  to  eliminate  requirement 

Materiel 

Low 

Improved  Flight  Control  System  (FCS)  -  Attitude 

Command  Attitude  Hold  (ACAH)  4-  Hover  Hold 

Materiel 

Medium 

Obstacle  warning  device  that  identifies  direction  of  ob¬ 
stacles  (virtual  rotor  disc) 

Materiel 

Medium 

Digital  Source  Collector  (DSC)  /Voice  recorder 

Materiel 

Medium 

Provide  pilot  capability  to  focus  outside  for  all  tasks 

(Day /night /NVG  HUD  with  weapons  and  flight  sym¬ 
bology) 

Materiel 

Medium 

Articulated  weapons  pylons 

Materiel 

High 

Improved  AN  VIS  resolution,  acuity,  and  FOV 

Materiel 

Low 

Crew  coordination  sustainment  training 

Training 

Low 

Field  a  high  fidelity  simulator  and  develop  accident 

avoidance  scinerio  for  simulator  training 

Materiel 

High 

Enhance  risk  management  policy  to  provide  feedback 

for  reassessments  to  commander  /decision  maker  as 

conditions  change 

Ldr  Dev 

VLow 

Increase  aircrew  experience  (flying  hour  program  and 

increase  proficiency  minimums)  -  increase  avg  exp  from 

400  hr  to  1000  hr 

Org 

High 

Redesign/standardize  fuel  hose  coupling 

Materiel 

Low 

Improve  qualification  (MOS)  and  unit  training  on  all 

aviation  refueling  equipment  (USAALS  coordination) 

Training 

VLow 

Establish  DA  program  to  procure  and  manage  stan- 

Materiel 

Low 

dardized  fueling  equipment  (need  to  research  propo¬ 
nent) 
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CONTROL 

CONTROL  STATEMENT 

DTLOMS 

COST 

OH58-C16 

Establish  standard  in  the  -10  to  disable  flight  controls 

when  the  crew  station  is  occupied  by  non-rated  pas¬ 
senger 

Doctrine 

Nil 

OH58-C17 

Develop  and  field  a  wire/obstacle  detection  system 

Materiel 

Medium 

OH58-C18 

Add  an  audio  warning  to  provide  a  throttle  warning 

at  400  feet  with  throttle  at  idle 

Materiel 

Medium 

OH58-C20 

Improve  autorot ational  characteristics/sink  rate 

Materiel 

High 

OH58-C21 

Establish  a  minimum  entry  altitude  for  initiation  of  a 

simulated  engine  failure 

Doct/Trg 

Nil 

OH58-C22 

Enhance  crew  coordination  task  (requiring  PI  to  con¬ 
firm  IP  has  rolled  throttle  on-check  throttle) 

Training 

Nil 

OH58-C23 

Increase  SEF  training(both  AQC,  IPC/MOI,  and  unit 

training) 

Training 

Low 

OH58-C24 

Modify  the  ATM  to  make  the  SEF  task  a  mandatory 

part  of  the  mission  briefing 

Training 

Nil 

OH58-C26 

Make  landing  gear  and  attachment  points  more  toler¬ 
ant  to  hard  landings 

Materiel 

Medium 

OH58-C27 

Increase  SEF/ Autorotation  training(both  AQC, 

IPC/MOI,  and  unit  training) 

Training 

Low 

OH58-C29 

Materiel  modification  already  applied 

Materiel 

Zero 

OH58-C30 

Enforce  maintenance  standards  with  command  em¬ 
phasis 

Ldr  Dev 

Nil 

OH58-C31 

Fill  the  OH-58D  maintenance  force  structure  (coordi¬ 
nate  with  USAALS/PERSCOM) 

Soldiers 

Low 

OH58-C33 

Incorporate  FADEC  fleetwide 

Materiel 

High 

OH58-C34 

Include  hazards  and  controls  in  MOI  flight  training 

guide 

Training 

Nil 

OH58-C35 

Enforce  procedures  with  command  emphasis 

Ldr  Dev 

Nil 
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CONTROL 

CONTROL  STATEMENT 

DTLOMS 

COST 

OH58-C36 

Modify  Fighter  Management  Procedures  to  include 

maintenance  personnel 

Ldr  Dev 

Nil 

OH58-C37 

Information  system  to  provide  realtime  weather  at  unit 

operations  (DTN,  WSI,  NOAA,  DUATS) 

Materiel 

Low 

OH58-C38 

Improved  aircraft  weather  detection  system 

Materiel 

High 

OH58-C39 

Increase  power  and  torque  available 

Materiel 

High 

OH58-C40 

Install  collective  soft  stop  (tactile  cueing  to  provide 

overtorque  feedback  to  pilot),  including  incident  re- 

construction  capability 

Materiel 

Medium 

OH58-C41 

Install  flight  envelop  cueing,  including  incident  recon¬ 
struction  capability 

Materiel 

Medium 

OH58-C42 

Ensure  retrofit  of  crashworthy  seats  is  applied  to  all 

aircraft 

Materiel 

Medium 

OH58-C43 

Equip  left  seat  collective  with  RPM  trim  switch,  search 

light  on-off,  and  search  light  control 

Materiel 

Medium 

OH58-C44 

Accelerate  ECP  application 

Materiel 

Low 

OH58-C45 

Design  a  reliable  fire  detection  and  suppression  system 

Materiel 

Medium 

OH58-C46 

Remove  SCAS  switch  from  cyclic 

Materiel 

High 

OH58-C47 

Resourcing  of  thermal  protective  equipment  for  all 

77F’s 

Materiel 

Medium 

OH58-C48 

Strengthen  Optical  Display  Assembly  training  (both 

AQC  and  unit  training) 

Training 

Low 

OH58-C49 

Increase  minimum  hover  altitude  in  unit  SOP  to  10 

feet  for  operating  on  rolling  ships 

Training 

Nil 

OH58-C50 

Identify  object  strike  hazards  effecting  hovering  alti¬ 
tudes  during  orientations,  reconnaissances  and  premis¬ 
sion  planning  for  incorporation  into  unit  SOPs  and 

premission  brefings. 

Training 

Nil 

OH58-C51 

Enforce  existing  standards  and  controls 

Ldr  Dev 

Nil 

OH58-C52 

Change  ATM  to  define  height  /altitude  restriction 

Training 

VLow 
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CONTROL 

CONTROL  STATEMENT 

DTLOMS 

COST 

OH58-C53 

Ensure  proper  marking  of  and  notification  of  poten¬ 
tial  hazards  is  provided  to  aircrews  and  updated  as 

necessary  during  flight  operations 

Ldr  Dev 

Nil 

OH58-C54 

Restrict  aircaft  flight  operations  in  periods  of  marginal 

weather  to  those  necessary  to  complete  the  mission 

Ldr  Dev 

Nil 

OH58-C55 

Include  appropriate  markings  to  indicate  aircraft  is  un¬ 
dergoing  maintenance 

Doct/TVg 

VLow 

OH58-C56 

Ensure  proper  IP  supervision  of  students 

Training 

Nil 

OH58-C57 

Ensure  objects  in  proximity  of  aircraft  (inside  of  bal¬ 
listic  barriers)  are  properly  secured 

Ldr  Dev 

VLow 

OH58-C58 

Inform  aircrew  of  hazards  associated  with  not  using 

provided  eye  protection  (visor) 

Training 

Nil 

OH58-C59 

Ensure  aviation  unit  maintenance  personnel  are  aware 

of  the  QDR  program  by  conductiing  initial  and  recur¬ 
ring  training 

Training 

Nil 

OH58-C60 

Ensure  SOPs  and  policies  for  precautionary  landings 

at  Ft  Rucker  are  reviewed  and  understood  by  Ft 

Rucker  personnel. 

Training 

VLow 

OH58-C61 

Ensure  information  on  engine  fuel  control  operation 

during  engine  starts  (including  degraded  modes  of  op¬ 
eration)  is  provided  in  the  operator’s  manual. 

Training 

VLow 

OH58-C62 

Ensure  complete  information  is  incorporated  in  the  op¬ 
erator’s  manual  addressing  cyclic  lock-out 

Training 

Nil 

OH58-C63 

Verify  communications  prior  to  flight 

Training 

Nil 

OH58-C64 

Develop  hover  training  progression  to  progress  to  stan¬ 
dard 

Training 

Nil 

OH58-C65 

Warning  to  inform  pilots  to  hover  into  the  wind  in  the 

event  of  an  emergency 

Training 

Nil 

OH58-C66 

Command  emphasis 

Ldr  Dev 

Nil 
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CONTROL 

CONTROL  STATEMENT 

DTLOMS 

COST 

OH58-C67 

Instill  self  discipline 

Ldr  Dev 

Nil 

OH58-C68 

Authorized  personnel  and  authorized  tools 

Org 

Low 

OH58-C69 

Warnings  in  technical  manuals 

Training 

Low 

OH58-C70 

Closer  attention  to  flight  by  tower  personnel/TSO 

Training 

Nil 

OH58-C72 

Emphasize  PPE  through  existing  force  protection 

Ldr  Dev 

Nil 

OH58-C73 

Provide  training  on  standardized  fire  supression  sys¬ 

Training 

Nil 

tems  for  tactical  FARP  operations 

OH58-C74 

Changes  to  unit  SOP  for  shutdown  and  post  flight 

inspections  after  conducting  emergency  procedures 

training. 

Training 

Nil 

OH58-C75 

Incorporate  changes  to  -10  to  highlight  upstop  condi¬ 

Training 

Low 

tions 

OH58-C76 

Education  of  effects  of  moisture/  dew  point 

Training 

Nil 

OH58-C77 

Research  Center  for  Army  Lessons  Learned  for  opera¬ 

Ldr  Dev 

Nil 

tional  risk  management  information 

OH58-C78 

Increase  frequency  of  wire  replacement 

Doctrine 

Nil 

OH58-C79 

Modify  WSPS  to  preclude  wire  cutter  contact  with 

ground  during  A/R  landings 

Materiel 

Low 

OH58-C80 

Provide  training  through  Pre-command  Course  to 

Ldr  Dev 

VLow 

increase  awareness  of  need  for  Bn  Commander  to 
broaden  junior  officer  development  -  management  of 
flying  hour  program  and  risk  management 

OH58-C81  Study  and  establish  minimum  operational  experience  Ldr  Dev  VLow 

and  flight  time  requirements  for  selection  as  aviation 
commander 

OH58-C82  Command  emphasis  campaign  from  HQDA(CSA),  Ldr  Dev  VLow 

Aviation  Branch  Chief,  and  Aviation  principals  to  en¬ 
force  standards 
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CONTROL 

CONTROL  STATEMENT 

DTLOMS 

COST 

H60-C01 

Add  mandatory  scenario  training  in  simulator  (emer¬ 
gency  procedures,  power  management,  brownout)  to 

include  resourcing  of  TDY 

Org 

Low 

H60-C02 

Book  of  hazards  and  controls 

Soldier 

VLow 

H60-C03 

Change  ATM  to  establish  new  flying  hour  category 

for  individual  task  flight  training  hours  (not  collective 

training)*  and  resource  individual  training. 

Doctrine 

Low 

H60-C04 

Develop  external  crashworthy  fuel  tanks  consider  suc¬ 
tion  fuel  system 

Materiel 

Medium 

H60-C05 

Establish  a  command  information  system  which  tracks 

all  forms  of  high  risk  behavior  and  marginal  perfor¬ 
mance 

Ldr  Dev 

Low 

H60-C06 

Establish  and  sustain  crewchief’s  ’’school  house”  train¬ 
ing  program 

Training 

Low 

H60-C07 

Establish  crew  coordination  sustainment  program 

Training 

Low 

H60-C08 

Establish  standards  for  and  resource  a  4th  crewmem¬ 
ber  for  high  workload  (mulit-ship,  night  )  operations. 

Include  in  mission  planning/briefs/assessment. 

Org 

Medium 

H60-C09 

Expand  AQC  training  (emergency  proce¬ 

dures/emergency  diagnosis,  mulitship  operations, 

and  flight  limitations) 

Training 

Low 

H60-C10 

Implement  a  change  to  the  flight  control  system  to  im¬ 
prove  aircraft  stability  and  control  in  low  speed  flight 

(Attitude  Command  Attitude  Hold) 

Materiel 

Medium 

H60-C11 

Increase  the  available  aircrew  experience. 

Org 

High 

H60-C12 

quirement  to  highlight  specific  controls  during  the  air 

Ldr  Dev 

Nil 

mission  brief. 
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CONTROL 

CONTROL  STATEMENT 

DTLOMS 

COST 

H60-C13 

Provide  commanders  a  better  ability  for  selection,  mis¬ 
sion  tailoring,  and  balancing  of  resources  to  do  the 

mission 

Ldr  Dev 

Low 

H60-C14 

Develop  a  wire  strike  protection  system  that  covers 

more  of  the  aircraft 

Materiel 

Medium 

H60-C15 

Enhance  NVG  field  of  view 

Materiel 

Medium 

H60-C16 

Develop  and  field  a  proximity  warning  system  (Virtual 

Rotor  Disk) 

Materiel 

Medium 

H60-C17 

Fund  and  install  flight  data  recorder  (FDR)  for  acci¬ 
dent  and  incident  investigation. 

Materiel 

Low 

H60-C18 

Increase  command  emphasis  (Advance  Course)  on 

safety  incentives 

Training 

VLow 

H60-C19 

Modify  AR  95-3  or  TC  1-210  to  require  2  hours  annu¬ 
ally  of  actual  instruments  for  each  PIC 

Training 

VLow 

H60-C20 

Develop  and  field  an  adjustable  proximity  warning  sys¬ 
tem/collision  avoidance. 

Materiel 

High 

H60-C21 

Develop  a  terrain  following  /  terrain  avoidance  radar 

Materiel 

High 

H60-C22 

Develop  standardized  training  support  package  for  use 

at  unit  level  targeted  on  ERFS  operations  to  include 

simulator  scenario  training,  jettison  stores,  a/c  perfor¬ 
mance  characteristics 

Materiel 

VLow 

H60-C23 

Manual  changes  to  describe  handling  characteristics 

Materiel 

VLow 

H60-C24 

Develop  and  install  new  Night  Vision  Systems 

Materiel 

Medium 

H60-C25 

Improve  aircraft  controllability  with  tanks  installed 

(pitch  bias  actuator,  digital  stabilator  amp) 

Materiel 

Low 

H60-C26 

Improve  IFR/IMC  infrastructure  in  selected  parts  of 

the  world  (ABSO  define) 

Materiel 

High 

H60-C27 

Wire  detection  system  using  laser  radar  or  HF  radar 

technology 

Materiel 

High 
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CONTROL  CONTROL  STATEMENT 


DTLOMS  COST 


H60-C28  Resource  aviation  maintenance  IAW  reference  XYX  to  Materiel  High 

match  requirements  of  complex  aircraft,  (link  to  US- 
AALS  needed  to  consider  dedicated  crews  to  aircraft) 

H60-C29  Full  authority  DEC  automatically  causes  engine  shut-  Materiel  High 

down 

H60-C30  Improve  engine  diagnostics  and  improve  cueing  of  cor-  Materiel  Medium 

relation  of  PCL  handle  to  engine 

H60-C31  Accelerate  addressing  materiel  failures  Materiel  High 

H60-C32  Improve  crew’s  ability  to  for  premission  planning  by  Materiel  Medium 

implementing  electronic  data  management  from  Air 
Warrior 

H60-C33  Enforce  rules  through  leadership  commitment  Ldr  Dev  VLow 

H60-C34  Digital  Source  Collector  (DSC)  and  envelope  cueing  Materiel  High 

(exceedences)  and  notice  to  pilot  of  exceedences/crew 
monitor 

H60-C35  Increase  minimum  distance  between  aircraft  for  mul-  Doctrine  Nil 

tiship  operations 

H60-C36  Modify  manual  to  establish  method  of  calculating  lat-  Materiel  Low 

eral  CG 

H60-C37  DAIG  evaluate  all  institutional  training  schools  for  in-  Training  Low 

tegration  of  risk  management  training  into  curriculum. 

H60-C38  Establish  or  enforce  selection  criteria  for  advanced  avi-  Org  VLow 

ator  training 

H60-C40  Develop  a  standardized  methodology  for  conducting  Ldr  Dev  Low 

mission  risk  assessments  with  the  objective  for  iden¬ 
tification  of  all  hazards  and  associated  controls  to  be 
presented  in  the  mission  brief 
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CONTROL 

CONTROL  STATEMENT 

DTLOMS 

COST 

H60-C41 

Evaluate  system  specific  instruction  in  the  UH-60  IP 

course.  Emphasize  what  system  specific  instruction  is 

imparted  by  IP’s  to  other  aviators  during  assignments 

(effectiveness  is  N/A) 

Training 

Low 

H60-C42 

Expand  leader  development  training  to  emphasize 

enforcement  of  aviation  maintenance  and  operations 

standards  (integration  into  advanced  course,  an  ex¬ 
portable  training  package,  and  modifying  BOC  and 

AOC) 

Ldr  Dev 

Low 

H60-C43 

Integrate  risk  management  training  into  aviation  of¬ 
ficer /WO/NCO  development  programs  (use  accident 

experience  as  part  of  the  training) 

Ldr  Dev 

Low 

H60-C44 

Install  Flight  Data  Recorders  (FDRs)  and  develop  pro¬ 
cedures  for  use  of  FDR  data  by  commanders  for  air¬ 
crew  training 

Materiel 

Low 

H60-C45 

Evaluate  H-60  maintenance  force  structure  (effective¬ 
ness  is  N/A) 

Org 

N/A 

H60-C46 

Investigate  improving  H-60  handling  qualities  by  im¬ 
plementing  strakes  to  improve  airflow  over  the  tail- 

boom  (N/A  effectiveness) 

Materiel 

Low 

H60-C47 

Develop  a  smooth  deflection  device  on  top  of  ALQ-144 

Materiel 

Low 

H60-C48 

Relocate  /redesign  ALQ-144  (substitute  ATIRCMS) 

Materiel 

Low 

H60-C49 

Develop  and  distribute  an  exportable  training  package 

for  unit  level  Aviation  refuelers  and  mandate  training 

requirement  prior  to  assignment  as  aviation  refueler 

Training 

Low 
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